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PREFACE 


The  Disability  Evaluation  System  (DES)  is  the  Department  of  Defense  management 
tool  used  to  determine  the  disposition  of  a  service  member  who  develops  a  physical 
or  medical  condition  that  calls  into  question  the  member’s  ability  to  perform  the  du¬ 
ties  of  his  or  her  office,  grade,  rank,  or  rating.  A  service  member  enters  the  DES  when 
a  medical  evaluation  calls  into  question  his  or  her  ability  to  meet  medical  retention 
standards  to  perform  military  duties.  A  member  who  does  not  meet  medical  reten¬ 
tion  standards  progresses  to  a  physical  disability  evaluation,  which  results  in  findings 
and  a  disposition  decision. 

The  Principal  Deputy  Assistant  Secretary  of  Defense  (Force  Management  Policy) 
asked  RAND  to  examine  how  training  DES  primary  participants  could  help  provide 
more-consistent  disability  evaluation  results  for  similarly  situated  members  of  the 
military  services.  This  report  documents  the  requested  DES  training  analysis  and 
recommended  changes  in  training  along  with  other  recommendations  to  improve 
system  performance. 

The  findings  in  this  report  should  be  of  primary  interest  to  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness,  the  Assistant  Secretary  of  Defense  for  Force 
Management  Policy,  the  Assistant  Secretary  of  Defense  for  Health  Affairs,  the 
Assistant  Secretary  of  Defense  for  Reserve  Affairs,  and  the  Secretaries  of  the  military 
departments.  This  report  should  also  be  of  interest,  to  varying  degrees,  to  the 
Surgeons  General,  commanders  of  military  treatment  facilities.  Medical  Evaluation 
Board  approving  authorities,  physicians  who  convene  Medical  Evaluation  Boards, 
physicians  who  refer  service  members  to  Medical  Evaluation  Boards  (referring 
physicians).  Physical  Evaluation  Board  (PEB)  approving  authorities,  PEB  members, 
PEB  Liaison  Officers  (PEBLOs),  patient  administrators  who  support  Medical 
Evaluation  Boards  and/or  PEBLOs,  PEB  administrative  action  oflScers,  appellate  re¬ 
view  board  members,  active  component  unit  commanders  and  Reserve  unit  com¬ 
manders  who  interact  with  the  DES,  and  attorneys  who  represent  service  members 
during  appeals  before  the  Formal  Physical  Evaluation  Board. 

This  research  was  conducted  for  the  Under  Secretary  of  Defense  for  Personnel  and 
Readiness  within  the  Forces  and  Resources  Policy  Center  of  RAND’s  National 
Defense  Research  Institute  (NDRI).  NDRI  is  a  federally  funded  research  and  de¬ 
velopment  center  sponsored  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff, 
the  unified  commands,  and  the  defense  agencies. 
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SUMMARY 


To  serve  in  the  United  States  military— the  Army,  Navy,  Marine  Corps,  Air  Force,  or 
Coast  Guard  (when  it  is  operating  as  a  service  within  the  Department  of  the  Navy) — 
service  members  must  meet  certain  medical  and  physical  standards  to  perform  du¬ 
ties  appropriate  to  their  office,  grade,  rank,  or  rating. 

A  medical  evaluation  may  call  into  question  a  service  member’s  ability  to  meet  medi¬ 
cal  retention  standards  to  perform  military  duties,  at  which  point  the  member  enters 
the  Disability  Evaluation  System  (DES).  Service  members  who  do  not  meet  medical 
retention  standards  progress  to  a  physical  disability  evaluation,  which  results  in  a 
disposition  decision. 

Primarily  while  undergoing  medical  and  physical  disability  evaluations,  the  service 
member  receives  counseling  regarding  what  to  expect  throughout  all  phases  of  the 
disability  evaluation  process,  the  significance  and  consequences  of  the  determina¬ 
tions  that  are  made,  and  his  or  her  rights,  benefits,  and  entitlements.  A  member  who 
disagrees  with  the  physical  disability  evaluation  findings  and  decision  may  redress 
that  disagreement  through  appellate  review. 

Appropriate  personnel  authorities  accomplish  final  disposition  of  the  service  mem¬ 
ber’s  case  by  issuing  orders  and  instructions  to  implement  the  determination  of  the 
respective  military  department’s  final  reviewing  authority.  The  service  member  exits 
the  DES  by  returning  to  duty,  separating  (with  or  without  compensation),  or  retiring 
for  disability  or  length  of  service. 

Title  10  of  the  U.S.  Code  (U.S.C.)  establishes  the  basis  for  disability  retirement  and 
separation.  The  DES  is  the  Department  of  Defense  (DoD)  management  tool  used  to 
determine  the  disposition  of  a  service  member  who  develops  a  physical  or  medical 
condition  that  calls  into  question  the  member’s  ability  to  perform  the  duties  of  his  or 
her  office,  grade,  rank,  or  rating.  The  DoD  assigns  responsibility  for  developing 
policies  to  implement  and  manage  the  DES  to  three  Assistant  Secretaries  of 
Defense — for  Force  Management,  Reserve  Affairs,  and  Health  Affairs.  All  three 
Assistant  Secretaries  report  to  the  Under  Secretary  of  Defense  for  Personnel  and 
Readiness. 

Disability  evaluation  training  was  first  recognized  in  1992  as  a  significant  factor  af¬ 
fecting  DES  performance.  At  that  time,  a  Department  of  Defense  Inspector  General 
audit  determined  that  the  DoD  DES  was  not  efficient  or  economical.  The  audit  re- 
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ported  that  disability  cases  were  not  processed  promptly  and  service  members  were 
incorrectly  rated  for  their  disabilities.  The  audit  report  noted,  among  other  things, 
that  military  personnel  who  adjudicated  disability  cases  were  inadequately  trained, 
resulting  in  inconsistent  application  of  disability  policy  and  the  lack  of  formal  train¬ 
ing  contributed  to  rating  deficiencies.  Several  other  reports  likewise  recommended 
that  the  Office  of  the  Secretary  of  Defense  (OSD)  focus  on  training  as  a  key  interven¬ 
tion  to  improve  the  performance  of  the  DES. 

DoD  disability  policy  documents  also  emphasize  the  role  of  training  in  achieving  the 
DES  goal  of  conducting  physical  disability  evaluation  in  a  consistent  and  timely 
manner.  The  policy  documents  assign  responsibility  for  developing  and  maintaining 
(1)  a  program  of  instruction  for  the  DES;  (2)  a  program  of  instruction  on  the  prepara¬ 
tion  of  Medical  Evaluation  Boards^  for  physical  disability  cases  to  be  used  by  military 
treatment  facilities  (MTFs);  and  (3)  a  program  of  instruction  on  the  medical  aspects 
of  physical  disability  adjudication,  to  include  the  application  of  the  Veterans 
Administration  Schedule  for  Rating  Disabilities  (VASRD)  for  use  by  PEB  adjudicators 
and  appellate  review  authorities. 

STUDY  COMMISSION 

In  1999,  the  Principal  Deputy  Assistant  Secretary  of  Defense  for  Force  Management 
Policy  asked  RAND’s  National  Defense  Research  Institute  to  identify  and  recommend 
changes  to  the  training  provided  to  primary  participants^  of  the  DES  to  ensure 
consistent  application  of  disability  policy  across  and  within  the  services. 

Consistent  application  of  disability  policy  is  one  means  of  addressing  unwarranted 
variability  in  differential  treatment  of  similarly  situated  service  members.  However, 
underlying  the  desire  to  reduce  variability  is  the  more  fundamental  objective  of 
enhancing  the  DES’s  ability  to  accomplish  its  purpose  and  desired  outcomes.  As  a 
result,  the  primary  question  addressed  in  this  study  became  the  following:  How  can 
changes  to  disability  evaluation  training  and  other  management  interventions 
improve  DES  performance? 

This  report  addresses  the  following  four  specific  research  tasks  for  the  Office  of  the 
Assistant  Secretary  of  Defense  for  Force  Management  Policy  that  focus  on  improving 
the  performance  of  the  DES: 


^To  avoid  misunderstanding,  we  avoid  using  the  acronym  MEB,  which  is  commonly  used  to  mean  three 
different  things:  the  group  of  physicians  who  convene  as  a  board,  the  narrative  summary,  and  the 
complete  disability  case  file.  Instead,  we  use  “Medical  Evaluation  Board"  when  referring  to  the  group  of 
physicians  who  convene  as  a  board  (including  those  who  pass  records  among  themselves  without  actually 
convening  a  board).  We  refer  to  the  actual  narrative  summary  as  the  “narrative  summary”  and  the 
disability  case  file  as  the  “medical  board." 

p 

For  the  purposes  of  this  study,  primary  participants  in  the  DES  include  the  following:  physicians  who  re¬ 
fer  service  members  to  Medical  Evaluation  Boards,  generally  known  as  referring  physicians;  physicians 
who  convene  Medical  Evaluation  Boards;  Medical  Evaluation  Board  approving  authorities;  PEB  Liaison 
Officers  (PEBLOs);  patient  administrators  who  support  Medical  Evaluation  Boards  and/or  PEBLOs;  PEB 
administrative  action  officers;  PEB  members;  PEB  approving  authorities;  post-PEB  appellate  review  board 
members;  active  component  unit  commanders;  Reserve  unit  commanders  who  determine  eligibility  for 
temporary  incapacitation  pay;  and  attorneys  who  represent  service  members  during  appeals  before  the 
Formal  Physical  Evaluation  Board. 
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•  describe  a  basis  for  assessing  the  performance  of  the  DBS 

•  identify  issues  affecting  the  current  performance  of  the  DBS  and  interventions  to 
resolve  those  issues 

•  analyze  DBS  training  needs  and  recommend  changes  in  training  to  improve  sys¬ 
tem  performance 

•  develop  a  method  for  continuously  monitoring  performance  of  the  DBS  over 
time. 

STUDY  APPROACH 

To  recommend  changes  to  the  training  provided  to  DBS  primary  participants  that 
would  ensure  consistent  application  of  disability  policy  across  and  within  the  ser¬ 
vices,  we  first  identified  a  number  of  instances  of  variability  in  policy  application 
across  and  within  the  military  departments.  We  captured,  and  then  anaiyzed,  those 
instances  of  variability  in  the  form  of  issues.  We  next  employed  an  issues-driven, 
bottom-up  “Goal  Babric”  analytic  methodology  (discussed  in  Chapter  4)  to  identify 
actions  needed  to  resolve  the  identified  issues  and  organize  those  actions  into  an 
overarching  plan  to  ensure  consistent  application  of  disability  policy  across  and 
within  the  services.  One  of  the  ten  categories  of  interventions  that  resulted  from  this 
approach  was  a  training  intervention  that  focused  on  resolving  current  performance 
issues. 

In  recognizing  that  consistent  application  of  disability  policy  is  just  one  means  of  ad¬ 
dressing  unwarranted  variability  in  differential  treatment  of  similarly  situated  service 
members,  we  also  focused  on  overall  DBS  performance.  To  focus  on  system  perfor¬ 
mance,  we  employed  a  purpose-driven,  top-down  approach  and  developed  a  state¬ 
ment  of  purpose^  and  desired  outcomes.^  This  approach  and  statement  serve  as 
guideposts  for  developing  a  comprehensive  disability  evaluation  training  interven¬ 
tion  and  a  management  information  system  to  monitor  the  effectiveness  of  the 
recommended  training  program  and  overall  system  performance  over  time. 

FINDINGS  AND  RECOMMENDATIONS 

The  major  findings  presented  in  this  report  cover  four  areas:  (1)  developing  a  basis 
for  assessing  DBS  performance;  (2)  identifying  issues  of  variability  in  DBS  policy  ap¬ 
plication  and  recommended  interventions;  (3)  conducting  a  DBS  training  analysis 
and  recommending  changes  in  training  to  improve  system  performance;  and  (4)  de¬ 
veloping  a  recommended  method  for  continuously  monitoring  DBS  performance. 


purpose  statement  describes  the  fundamental  and  unchanging  reason  the  DBS  exists.  It  differentiates 
the  DBS  from  other  human-resource  management  systems  (and  tools). 

'^Desired  outcomes  explicitly  describe  the  intended  results  of  operating  the  system  to  achieve  its  stated 
purpose — the  results  that  matter  to  DBS  customers. 
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A  Basis  for  Assessing  DES  Performance 

To  assess  any  system’s  performance,  it  is  first  necessary  to  understand  the  system’s 
fundamental  purpose  and  its  desired  outcomes.  The  system’s  stated  purpose  is  the 
foundation  for  designing,  redesigning,  organizing,  and  monitoring  every  aspect  of 
the  system. 

We  found  that  no  shared  statement  of  purpose  for  the  DoD  DES  existed;  therefore, 
we  constructed  a  DES  purpose  statement  and  a  set  of  desired  system  outcomes  in 
order  to  develop  our  recommendations.  We  determined  that  the  DES  exists  to  evalu¬ 
ate  service  members  with  potentially  unfitting  conditions^  in  a  fair,  consistent,  effi¬ 
cient,  and  timely  manner  and,  likewise,  to  remove  those  unable  to  fulfill  the  duties  of 
their  office,  grade,  rank,  or  rating,  and  determine  a  disability  rating  percentage  for 
those  removed. 

We  likewise  constructed  the  following  set  of  desired  system  outcomes: 

1.  Service  members  having  a  similar  condition  and  similar  office,  grade,  rank,  or 
rating  receive  similar  fitness  decisions  within  the  military  department. 

2.  Service  members  found  unfit  receive  similar  disability  ratings  for  similar 
conditions  across  and  within  the  military  departments. 

3.  Service  members  freely  and  appropriately  exercise  their  rights  to  administrative 
due  process. 

4.  Service  members  return  to  duty,  or  separate  or  retire  for  disability,  in  a  timely 
manner. 

5.  Primary  participants  perform  their  duties  as  efficiently  as  possible  so  that,  col¬ 
lectively,  they  return  service  members  to  duty,  or  separate  or  retire  them  for  dis¬ 
ability,  in  a  fair,  consistent,  and  timely  manner. 

Because  a  common,  shared  purpose  and  set  of  desired  outcomes  do  not  currently 
exist,  reaching  consensus  on  these  constructs  is  an  important  first  step  in  the  devel¬ 
opment  of  interventions  to  improve  the  performance  of  the  DES.  Therefore,  we  rec¬ 
ommend  that  the  Assistant  Secretary  of  Defense  for  Force  Management  Policy 
(ASD/FMP)  develop  a  statement  of  purpose  and  desired  outcomes  for  the  DES  to 
serve  as  the  basis  for  the  DoD  DES  training  program. 

Issues  of  Variability  in  DES  Policy  Application  and  Recommended 
Interventions 

We  identified  43  issues— regarding  variability  in  policy  application  across  or  within 
the  military  departments  or  problems  identified  by  primary  participants — that  effect 
the  performance  of  the  DES.  The  issues-based,  bottom-up  analysis  suggested  the 


^Unfitting  condition  is  a  term  commonly  used  in  DES  policy.  Although  not  explicitly  dehned  in  DoD  or 
U.S.  statutory  documents,  the  term  could  be  said  to  refer  to  “a  medical  condition  resulting  from  disease  or 
injury  that  makes  a  service  member  unfit  to  perform  the  duties  of  the  member’s  office,  grade,  rank,  or 
rating”  (DoD  Directive  1332.18, 1996,  p.  2;  DoD  Directive  6130.3,  1994,  p.  1). 
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following  ten  categories  of  broad-based  interventions  consisting  of  specific  actions 
for  resolving  the  various  issues  we  identified.  We  recommend  that  the  DBS 
leadership  adopt  and  act  upon  these  interventions  in  the  context  of  a  near-term  plan, 
as  detailed  in  Chapter  4: 

•  Assistant  Secretary  of  Defense  Decisions 

•  Policy  Guidance 

•  Organizational  Change 

•  Personnel  Policy 

•  Personnel  Management 

•  Training 

•  Information  Source  Development 

•  Management  Information  System  Deployment 

•  Process 

•  Incentives. 

Because  the  recommended  interventions  are  based  on  reported  or  observed  in¬ 
stances  of  policy  application — information  that  is  not  necessarily  complete,  objec¬ 
tive,  or  empirically  based — ^we  expect  that  die  interventions  are  not  as  finely  tuned  as 
they  otherwise  might  be. 

DBS  Training  Analysis  and  Recommended  Changes  in  Training  to  Improve 
System  Performance 

For  the  third  research  task  in  this  study,  we  present  a  comprehensive  training  needs 
analysis  and  training  recommendations  to  improve  DBS  performance. 

We  identified  12  primary  participant  populations  who  require  specific  bodies  of 
knowledge  and  skills  to  execute  disability  policy  throughout  the  military  depart¬ 
ments.  We  examined  various  aspects  of  the  target  training  population,  such  as  char¬ 
acteristics,  turnover,  geographic  dispersion,  subject-matter  expertise  and  variation  in 
levels  of  required  disability  evaluation  expertise,  and  computer  literacy. 

Given  the  existing  job  designs  for  the  primary  participant  populations,  we  translated 
the  DBS  purpose  and  desired  outcome  statements  that  we  constructed  into  state¬ 
ments  of  general  competencies  for  10  of  the  12  primary  participant  populations. 
Those  primary  participant  populations,  and  their  respective  competencies,  are  as 
follows: 

Physicians  Who  Write  Narrative  Summaries 

•  are  able  to  determine  the  appropriate  diagnosis 

•  are  able  to  determine  if  a  service  member’s  condition  calls  into  question  his  or 
her  ability  to  meet  medical  retention  standards 
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•  are  able  to  synthesize  a  service  member’s  medical  evidence  from  all  appropriate 
consultations  into  a  single  narrative  summary  that  contains  sufficient  informa¬ 
tion  in  the  appropriate  format  for  a  PEB  to  adjudicate  the  case. 

PEBLOs  and  Disability  Evaluation  Counselors 

•  are  able  to  advise  service  members  on  the  DES  process;  their  rights,  benefits,  and 
entitlements;  and  what  to  expect  as  the  service  member’s  medical  board 
progresses  through  the  DES 

•  are  able  to  gather  and  process  patient  information  to  assemble  medical  boards 
(case  files)  that  contain  sufficient  information  in  the  appropriate  format  for  a 
PEB  to  adjudicate  the  case. 

Patient  Administrators 

•  are  able  to  assist  Medical  Evaluation  Boards  and  PEBLOs  in  gathering  and  pro¬ 
cessing  patient  information  to  assemble  medical  boards  that  contain  sufficient 
information  in  the  appropriate  format  for  a  PEB  to  adjudicate  the  case. 

Medical  Evaluation  Board  Members 

•  are  able  to  determine  whether  the  medical  board  includes  appropriate  specialty 
consultations  with  sufficient  information 

•  are  able  to  determine  the  duty  limitations  associated  with  the  diagnosis 

•  are  able  to  determine  whether  the  service  member  meets  the  military  depart¬ 
ment’s  medical  retention  standards  for  continued  military  duty. 

Medical  Evaluation  Board  Approving  Authorities 

•  are  able  to  identify  complete  and  accurate  medical  boards. 

PEB  Administrative  Action  Officers 

•  are  able  to  ensure  that  contents  of  medical  boards  received  by  the  PEB  are  com¬ 
plete  and  accurate  for  adjudication 

•  are  able  to  obtain  missing  information,  monitor  and  move  medical  boards 
through  the  system,  and  exchange  information  with  PEBLOs. 

Physical  Evaluation  Board  Members 

•  are  able  to  apply  disciplined  military  department  fitness  standards  in  a  uniform 
manner 

•  are  able  to  apply  other  rules  uniformly  such  that  members  having  similar  condi¬ 
tions  and  a  similar  office,  grade,  rank,  or  rating  receive  similar  disability  ratings 
across  and  within  the  military  departments 

•  are  able  to  document  the  substantial  evidence  that  supports  all  PEB  decisions. 
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Physical  Evaluation  Board  Approving  Authorities 

•  are  able  to  identify  consistent  application  of  military  department  fitness  stan¬ 
dards  such  that  members  having  similar  conditions  and  similar  office,  grade, 
rank,  or  rating  receive  similar  fitness  decisions  within  the  military  department. 

•  are  able  to  identify  consistent  application  of  other  rules  such  that  members  hav¬ 
ing  similar  conditions  receive  similar  disability  ratings  across  and  within  the 
military  departments 

•  are  able  to  identify  sufficient  documentation  of  the  substantial  evidence  that 
supports  all  PEB  decisions. 

Post-PEB  Appellate  Review  Board  Members 

•  are  able  to  apply  disciplined  military  department  fitness  standards  in  a  uniform 
manner 

•  are  able  to  apply  other  rules  uniformly  such  that  members  having  a  similar 
condition  receive  similar  disability  ratings  across  and  within  the  military 
departments 

•  are  able  to  document  the  substantial  evidence  that  supports  all  decisions. 

Unit  Commanders 

•  are  able  to  provide  written  evidence  with  sufficient  detail  for  PEB  consideration 
that  documents  their  judgment  of  how  a  service  member’s  medical  condition 
impacts  the  member’s  ability  to  perform  the  duties  of  his  or  her  office,  grade, 
rank,  or  rating,  and  specifically  how  the  condition  impacts  his  or  her  ability  to 
deploy,  and  whether  there  are  any  pending  adverse  actions  against  the  service 
member. 

The  desired  system  outcomes  not  only  shape  the  performance  competencies  for  in¬ 
dividuals  assigned  to  the  DES,  they  suggest  specific  knowledge  necessary  for  physi¬ 
cians  who  refer  service  members,  unit  commanders  (both  active  and  Reserve)  who 
interact  with  the  DES,  and  attorneys  who  advise  and  represent  members.  The  desired 
outcomes  point  to  a  DoD  training  emphasis  on  DES  topics  and  skills  in  applying 
knowledge  of  those  topics  across  the  military  departments.  Likewise,  the  OSD’s  focus 
on  consistent  policy  application  suggests  that  DES  topics  and  the  associated  skills 
required  to  apply  knowledge  of  those  topic  areas  are  the  most  relevant  aspects  of  a 
DoD  training  intervention. 

As  a  result,  we  compiled  a  comprehensive  list  of  DES  training  topics  from  policy  doc¬ 
uments  and  military  departments’  current  training  syllabi.  We  associated  each  sug¬ 
gested  topic  with  the  primary  participant  populations  who  require  knowledge  of  that 
topic  to  produce  the  desired  on-the-job  results,  recognizing  that  different  popula¬ 
tions  may  apply  the  same  knowledge  differently  in  their  respective  jobs. 

The  proposed  competencies,  together  with  the  analysis  of  required  primary  partici¬ 
pant  population  knowledge  of  specific  DES  topics  to  achieve  desired  on-the-job  re- 
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suits,  indicates  that  the  primary  participant  populations  require  different  levels  of 
knowledge  for  many  of  the  same  DES  topics.  We  sorted  the  primary  participant 
populations  that  require  essentially  the  same  level  of  knowledge  of  the  same  set  of 
DES  topics  into  five  population  clusters,  as  shown  in  Table  S.l. 

Further  analysis  suggested  designing  the  training  content— the  DES  topics  for  in¬ 
struction — as  five  distinct  training  packages,  one  per  population  cluster. 

Other  considerations  in  addition  to  the  system’s  stated  purpose,  desired  outcomes, 
competencies,  and  content  affect  training  design.  To  inform  the  format  and  timing  of 
recommended  training,  we  assessed  the  following  considerations:  DES-specific 
knowledge  often  needed  immediately  upon  assignment;  frequency  of  use  of  DES 
topics  within  the  primary  participant  populations’  bodies  of  knowledge;  assignment 
practices  that  cause  high  turnover  rates  among  some  primary  participant  popula¬ 
tions;  and  military  departments’  DES  Web  sites. 

We  observed  a  common  cultural  trait  across  all  the  military  departments— a  high 
commitment  to  excellence  in  training,  regardless  of  the  training  method.  Numerous 
studies  in  the  training  literature  report  “no  significant  difference"  in  learning  results 
between  self-directed  computer-based  distance  training  and  traditional  classroom 
training.  We  compared  current  military  department  training  practices  with  the  pro¬ 
posed  training  packages  (developed  and  monitored  by  the  OSD)  and  analyzed  the 
advantages  and  disadvantages  of  designing  the  DES  training  program  as  a  self- 
directed,  computer-based  distance-training  program  or  as  a  classroom-training  pro¬ 
gram.  Based  on  our  analysis,  we  recommend  that  the  Office  of  the  ASD/FMP  develop 
and  monitor  knowledge-based  training  in  which  the  content  focuses  on  the  sug¬ 
gested  list  of  DES  topics  that  collectively  constitute  a  specific  body  of  knowledge  for 
each  primary  participant  population  cluster.  We  further  recommend  delivering  this 
knowledge-based  training  through  a  Web  site  devoted  to  disability  evaluation  train¬ 
ing,  which  is  made  accessible  to  all  primary  participants. 


Table  s.l 

Primary  Participant  Population  Clusters 

Population  Cluster  1 

PEBLOs  and  disability  evaluation  counselors 

Patient  administrators 

PEB  administrative  action  officers 

Population  Cluster  2 

Physicians  who  write  narrative  summaries 

Medical  Evaluation  Board  members 

Medical  Evaluation  Board  approving  authorities 

Population  Cluster  3 

PEB  members 

PEB  approving  authorities^ 

Appellate  Review  Board  members 

Attorneys  who  represent  and  advise  service  members 

Population  Cluster  4 

Active  component  unit  commanders 

Population  Clusters 

Reserve  component  commanders 

These  authorities  are  the  Deputy  Commander,  Army  Physical  Disability  Agency;  President,  Physical 
Evaluation  Board  (Department  of  the  Navy);  and  Chief,  Air  Force  Physical  Disability  Division. 
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This  self-directed  computer-based  distance  training  is  a  basic  course  in  disability 
evaluation.  See  Table  5.2  in  Chapter  5,  which  organizes  a  comprehensive  list  of  DBS 
topics  into  five  distance-training  packages,  each  designed  to  meet  the  training  needs 
of  a  particular  cluster  of  primary  participant  populations.  The  table  further  organizes 
the  topics  roughly  in  descending  order  of  common  training  needs  across  population 
clusters,  starting  with  those  topics  that  all  population  clusters  require  knowledge  of, 
and  ending  with  those  topics  required  by  only  one  cluster. 

All  five  population  clusters  require  knowledge  of  many  of  the  same  DBS  topics, 
although  different  population  clusters  need  to  know  how  to  apply  some  topics  in 
different  ways  to  achieve  their  specific,  desired  on-the-job  results.  Although  the  dif¬ 
ferent  training  packages  contain  many  of  the  same  DBS  topics,  the  learning  objec¬ 
tives,  content  presentation,  and  criterion  referencing®  should  match  the  specific  job 
application  needs  of  each  target  population  cluster  (and  some  learning  objectives, 
content  presentation,  and  criterion  referencing  will  be  the  same  for  different  popu¬ 
lation  clusters). 

We  assumed  that  the  OSD  develops  a  Web  site  devoted  to  disability  evaluation 
training  and  establishes  the  recommended  self-directed,  computer-based  distance¬ 
training  packages.  We  further  assumed  the  training  packages  “teach"  the  DBS  bodies 
of  knowledge  to  the  degree  intended.  We  then  asked  the  following  question:  Do  pri¬ 
mary  participants  require  additional  training  to  apply  policy  consistently  across  and 
within  military  departments  to  produce  the  desired  on-the-job  results? 

To  answer  this  question,  we  reexamined  the  proposed  primary  participant  compe¬ 
tencies.  The  competency  statements  suggest  that  PBB  members,  PBB  approving 
authorities,  and  post-PBB  appellate  review  board  members  across  military  depart¬ 
ments,  in  particular,  stand  to  benefit  from  collaboration  with  peers  on  how  to  uni¬ 
formly  apply  the  rules,  procedures,  and  other  considerations  in  determining  fitness, 
assigning  the  VASRD  or  analogous  codes,  and  assigning  disability  ratings. 

Likewise,  Medical  Bvaluation  Board  members  and  approving  authorities  across  mili¬ 
tary  departments  stand  to  benefit  from  collaboration  with  peers  in  how  to  apply  dis¬ 
ciplined  medical  retention  standards  uniformly,  such  that  members  having  a  similar 
condition  and  similar  office,  grade,  rank,  or  rating  receive  similar  medical  retention 
decisions.  Attorneys  who  advise  and  represent  members  are  also  likely  to  benefit 
from  collaboration  with  Medical  Bvaluation  Board  members  and  approving 
authorities,  PBB  members  and  approving  authorities,  and  appellate  review  board 
members  across  military  departments  during  classroom  training. 

We  recommend  supplementing  the  DoD  self-directed  computer-based  distance¬ 
training  packages  with  DoD  traditional  classroom  training  for  PBB  members  and 
approving  authorities,  post-PBB  appellate  review  board  members,  and  Medical 
Bvaluation  Board  members  and  approving  authorities  across  the  military  depart- 


^Criterion  referencing  refers  to  the  method  of  testing  that  is  most  often  used  in  self-directed  computer- 
based  distance  training.  The  test  questions  are  written  directly  from  the  stated  learning  objectives  and  can 
be  answered  directly  from  the  material  presented.  In  other  words,  criterion-referenced  tests  contain  no 
hidden  meanings  or  trick  questions. 
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merits.  The  classroom  training  focuses  on  applying  a  particular  set  of  DES  topics  to 
develop  the  skills  necessary  to  evaluate  and  adjudicate  cases  and  apply  disability 
policy  consistently  across  and  within  the  military  departments.  The  classroom 
training  is  designed  explicitly  to  supplement  the  self-directed  computer-based  dis¬ 
tance  training.  As  such,  completing  the  appropriate  distance-training  package  is  a 
prerequisite  for  enrolling  in  classroom  training,  evidenced  perhaps  by  a  certificate  of 
self-certified  mastery  of  the  required  knowledge  and  skills.  Learning  objectives,  con¬ 
tent,  and  student  learning  evaluation  differ  from  the  distance-training  packages  in 
that  they  focus  on  applying  a  particular  set  of  the  DES  topics  learned  in  the  distance¬ 
training  packages  to  a  variety  of  real-life  cases. 

The  DES  topics  shown  in  Table  5.3  in  Chapter  5  form  the  basis  for  classroom-training 
content.  That  set  of  DES  topics  enables  students  to  practice  applying  the  numerous 
standards,  rules,  procedures,  and  other  considerations  to  a  wide  variety  of  case 
studies  in  a  controlled  classroom  environment  in  which  students  collaborate  on 
making  decisions  that  result  in  consistent  dispositions. 

Both  the  self-directed  computer-based  distance-training  packages  and  the 
classroom-training  package  rely  on  experienced  and  credible  subject-matter  experts 
who  are  able  to  develop  and  deliver  high-quality  training  that  produces  the  desired 
on-the-job  results  from  the  trained  populations.  These  subject-matter  experts  serve 
as  adjunct  faculty  who  are  delegated  authority  by  the  OSD  to  develop  and  deliver  the 
DoD  disability  evaluation  training. 

The  self-directed  computer-based  distance-training  packages  and  the  classroom¬ 
training  package  are  based  on  the  system  purpose  and  desired  outcomes  that  in¬ 
formed  the  primary  participant  competencies  proposed  in  this  report. 

Like  the  suggested  statements  of  DES  purpose,  desired  system  outcomes,  and  pri¬ 
mary  participant  competencies,  the  training  analysis  and  the  resulting  training  pack¬ 
ages  are  presented  as  a  template,  or  a  starting  point,  for  consideration  by  the  Office 
of  the  ASD/FMP,  in  consultation  with  the  Assistant  Secretary  of  Defense/Health 
Affairs  (ASD/HA  )and  the  Assistant  Secretary  of  Defense/ Reserve  Affairs  (ASD/RA), 
and  representatives  of  the  military  departments’  PEBs  and  Office  of  the  Surgeons 
General. 

We  derived  the  suggested  DES  purpose  statement  and  set  of  DES  desired  outcomes 
from  DoD  and  military  department  documents  and  from  interviews  with  primary 
participants.  As  a  result,  they  should  be  generally  acceptable  to  decisionmakers  in 
the  OSD  and  military  departments;  however,  we  did  not  attempt  to  secure  agreement 
from  those  decisionmakers.  Rather,  we  believe  it  is  essential  for  the  ASD/FMP,  in 
consultation  with  the  ASD/RA  and  ASD/FIA,  to  decide  on  a  stated  DES  purpose  and 
set  of  desired  outcomes,  using  our  proposed  framework  as  a  starting  point.  The  ob¬ 
jective  of  the  ASDs’  deliberations  is  a  common  framework  for  developing  a  sense  of 
ownership  of  the  DES  purpose  and  desired  outcomes— the  purpose  and  desired  out¬ 
comes  inform  all  other  decisions  and  interventions. 

Whatever  statement  of  DES  purpose  and  desired  system  outcomes,  and  statements 
of  primary  participant  competencies,  are  decided  upon  should  form  the  basis  for 
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conducting  a  comprehensive  training  needs  assessment  which  should,  in  turn,  in¬ 
form  development  of  training  packages  to  enable  the  primary  participants  to  pro¬ 
duce  the  desired  on-the-job  results. 

If  a  later  assessment  of  training  effectiveness  demonstrates  that  these  training  pack¬ 
ages  do  not  enable  the  primary  participant  populations  to  produce  the  desired  on- 
the-job  results,  the  OSD  should  modify  the  training  packages  so  that  they  perform  as 
intended. 

We  estimate  that  this  training  program  (the  combined  self-directed  computer-based 
distance-training  packages  and  the  classroom-training  package)  will  cost  approxi¬ 
mately  $12.8  million  for  a  five-year  training  time  frame  (the  majority  of  this  cost  is 
the  opportunity  cost  for  course  participation).  But,  the  DoD  can  accrue  an  estimated 
$15.2  million  in  quantifiable  benefits  from  this  program  in  addition  to  a  variety  of 
nonquantifiable  benefits. 

A  Method  for  Continuously  Monitoring  DBS  Performance 

To  evaluate  a  system  and  improve  its  performance,  it  is  necessary  to  have  a  system¬ 
atic  method  for  tracking  how  well  the  system  is  functioning.  We  developed  a  number 
of  performance  measures  (and  metrics  that  support  those  measures)  that  can  be 
used  to  monitor  how  well  the  DBS  meets  external  customer  expectations,  which  we 
defined  in  terms  of  the  purpose  and  outcomes  of  the  DBS.  The  DBS  exists  to  serve 
two  categories  of  external  customers:  service  members  and  individual  military  ser¬ 
vices.  For  service  members,  expectations  center  on  similar  dispositions  (among  ser¬ 
vice  members  in  similar  circumstances)  and  due  process.  For  the  military  services, 
expectations  center  on  expeditious  processing  and  efficient  operations. 

The  performance  measures  we  developed  encompass  direct  customer  perceptions  of 
how  well  the  DBS  meets  their  expectations  and  indirect,  but  more  objective  and 
quantitative,  measures  of  performance.  The  indirect  measures  include  outcome,  out¬ 
put,  and  input  measures  that  are  linked  in  a  framework  that  identifies  the  relation¬ 
ship  among  the  measures  and  how  they  affect  overall  system  performance.  Outcome 
measures  include  case  variability,  number  of  appeals,  time  to  replace  an  unfit  service 
member,  and  total  system  cost.  Output  measures  include  percentage  of  primary  par¬ 
ticipants  certified,  productivity,  cost  per  case,  average  processing  time,  number  of 
reworks,  and  time  to  promulgate  policy  change.  The  sole  input  measure  is  total  re¬ 
sources. 

We  recommend  that  the  OSD  develop  and  maintain  a  comprehensive  management 
information  system  capable  of  monitoring  relevant  performance  measures  that  en¬ 
able  leaders  to  assess,  analyze,  and  take  action  to  continuously  improve  the  perfor¬ 
mance  of  the  DBS.  We  further  recommend  that  the  OSD  summarize  the  information 
gleaned  from  the  data,  which  are  gathered  and  analyzed,  and  share  that  information 
with  DBS  primary  participants  so  that  they  may  also  act  on  it  to  continuously  im¬ 
prove  DBS  performance. 
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Chapter  One 

INTRODUCTION 


In  accordance  with  Chapter  61  of  Title  10,  U.S.  Code  (U.S.C.),  Department  of  Defense 
(DoD)  policy  established  the  Disability  Evaluation  System  (DES)  as  the  mechanism 
for  implementing  retirement  or  separation  of  a  military  member  due  to  physical 
disability.  The  system  consists  of  four  elements:  medical  evaluation;  physical 
disability  evaluation,  to  include  appellate  review;  counseling;  and  final  disposition.^ 

A  service  member  can  potentially  experience  the  system  in  four  progressive  phases: 

1.  A  medical  evaluation  of  a  potentially  disabling  condition 

2.  A  physical  disability  evaluation  and  an  opportunity  to  appeal  findings  and  rec¬ 
ommendations  before  a  formal  hearing 

3.  Two  or  three  higher-level  appellate  reviews 

4.  Final  disposition  (return  to  duty,  separation,  or  retirement). 

The  service  member  receives  counseling  as  needed  throughout  the  process,  most  in¬ 
tensively  during  the  first  two  phases. 

DoD  policy,  in  accordance  with  Title  10,  U.S.C.,  requires  consistent  and  equitable 
application  of  standards  for  all  determinations  related  to  physical  disability  evalua¬ 
tion  of  active  component  and  Ready  Reserve  service  members  (DoD  Instruction 
1332.38,  1996,  Sec.  4.3).  DoD  policy  also  requires  the  Secretaries  of  the  military 
departments  to  manage  the  military  department-specific  DES  to  ensure  uniform 
interpretation  of  disability  policies  and  procedures  (DoD  Instruction  1332.38,  1996, 
Sec.  5.5.2)  and  uniform  application  of  the  governing  laws  and  DoD  policy  (DoD 
Directive  1332.18, 1996,  Sec.  4.4.3). 

The  Principal  Deputy  Assistant  Secretary  of  Defense  (ASD)  for  Force  Management 
Policy  (FMP)  asked  RAND’s  National  Defense  Research  Institute  to  identify  and 
recommend  changes  to  the  training  provided  to  the  primary  participants  of  the  DES 
to  ensure  the  consistent  application  of  disability  policy,  across  and  within  the  mili¬ 
tary  services:  the  Army;  the  Navy,  including  the  Coast  Guard  when  it  is  operating  as  a 


^Although  final  disposition  is  an  element  of  the  DES,  it  is  carried  out  by  the  personnel  functions  of  the 
military  departments,  largely  in  the  context  of  policies  unrelated  to  the  DES.  Consequently,  this  report 
focuses  on  medical  evaluation,  physical  disability  evaluation,  and  counseling. 
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military  service  in  the  Navy;  the  Marine  Corps;  and  the  Air  Force.  For  the  purposes  of 
this  report,  primary  participants  in  the  DBS  include 

•  physicians  who  refer  service  members  to  Medical  Evaluation  Boards,  generally 
known  as  referring  physicians 

•  physicians  who  convene  Medical  Evaluation  Boards 

•  Medical  Evaluation  Board  approving  authorities 

•  Physical  Evaluation  Board  Liaison  Officers  (PEBLOs)^ 

•  patient  administrators  who  support  Medical  Evaluation  Boards  and/or  PEBLOs 

•  Physical  Evaluation  Board  (PEB)  administrative  action  officers 

•  PEB  members 

•  PEB  approving  authorities^ 

•  appellate  review  board  members 

•  active  component  unit  commanders 

•  Reserve  unit  commanders 

•  attorneys  who  represent  and  advise  service  members. 

This  report  communicates  to  the  Principal  Deputy  ASD/FMP  our  assessment  and 
recommended  changes  to  the  training  provided  to  primary  participants  of  the  DES  to 
ensure  consistent  application  of  disability  policy,  across  and  within  the  military  ser¬ 
vices.  Our  recommendations  are  based  on  research  conducted  between  January  1999 
and  January  2000. 

QUESTIONS  OF  INTEREST  REGARDING  THE  DISABILITY 
EVALUATION  SYSTEM 

During  early  meetings  with  the  DES  project  sponsor  and  later  meetings  with  the  DES 
primary  participants,  the  following  salient  questions  were  raised. 

•  What  does  consistent  disability  policy  application  "look  like”  across  and  within 
the  military  departments? 

•  What  are  the  desired  outcomes  or  results  of  consistent  policy  application  across 
and  within  the  military  departments? 

•  How  is  consistent  disability  policy  application  measured  across  and  within  the 
military  departments,  given  their  different  missions  and  requirements? 


^In  the  Department  of  the  Navy,  PEBLOs  are  also  knovim  as  Disability  Evaluation  Counselors. 

q 

These  authorities  include  the  Deputy  Commander,  Army  Physical  Disability  Agency;  President,  Physical 
Evaluation  Board  (Department  of  the  Navy);  and  Chief,  Air  Force  Physical  Disability  Division. 
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•  What  are  the  important  causes  or  sources  of  variability  in  the  application  of  dis¬ 
ability  policy  today? 

•  How  much  variability  is  desirable  and/or  acceptable  in  the  application  of  dis¬ 
ability  policy  across  and  within  the  military  departments,  given  their  different 
missions  and  requirements? 

•  What  are  the  most  effective  means  for  reducing  undesirable  and/or  unacceptable 
levels  of  variability? 

Unwarranted  variability  is  the  manifestation  of  a  problem— that  the  system  is  failing 
to  accomplish  its  purpose  and  desired  outcomes.  Consistent  application  of  disability 
policy  is  but  one  means  of  addressing  this  problem.  A  more  robust  solution  to  the 
problem  requires  a  broader  perspective.  Underlying  the  desire  to  reduce  variability  is 
the  more  fundamental  objective  of  enhancing  the  ability  of  the  DES  to  accomplish  its 
purpose  and  desired  outcomes.  As  a  result,  the  central  question  of  interest  now  is: 
How  can  training  and  other  interventions  to  improve  system  performance  enable  the 
DES  to  achieve  its  desired  ends? 

RESEARCH  APPROACH 

In  the  context  of  the  question,  how  can  training  and  other  interventions  enable  the 
DES  to  achieve  its  desired  outcomes,  our  specific  research  tasks  focused  on  improv¬ 
ing  the  performance  of  the  DES.  Those  tasks  include  the  following: 

•  Describing  a  basis  for  assessing  the  performance  of  the  DES 

•  Identifying  major  issues  affecting  the  current  performance  of  the  DES  and  rec¬ 
ommending  interventions  to  resolve  those  issues 

•  More  specifically,  assessing  existing  training  programs  in  terms  of  their  effect  on 
performance  of  the  DES,  and  proposing  and  evaluating  recommended  changes 
in  content,  delivery  method,  and  timing  of  the  training  to  improve  performance 

•  Developing  a  process  for  monitoring  the  performance  of  the  DES  and  training 
effectiveness  over  time. 

We  reviewed  the  governing  U.S.  statutes,  DoD  disability  policy  documents,  and  the 
military  departments’  disability  policy  documents.  We  interviewed  numerous  diverse 
primary  participants  of  the  DES.  In  addition,  throughout  1999,  we  observed  the 
military  departments’  major  disability  evaluation  training  events — including  the 
annual  Navy  Physical  Evaluation  Board  Liaison  Officer  Conference,  the  twice-yearly 
Air  Force  Physical  Evaluation  Board  Liaison  Officer  Training,  the  annual  Army 
Worldwide  Physical  Evaluation  Board  Liaison  Officer  Conference,  and  the  annual 
Army  Adjudicators’  Course — all  of  which  were  provided  with  PEB  resources  from  the 
personnel  community.  We  also  applied  core  concepts  from  the  performance 
measurement,  training,  and  strategic-management  literature. 

Because  no  commonly  shared  statement  of  system  purpose  or  desired  DES  outcomes 
exist  across  the  military  departments,  except  for  a  narrow  focus  on  timeliness  per- 
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formance  standards,  we  employed  two  approaches  to  study  the  system.  First,  we 
employed  a  bottom-up,  issues-driven  approach  in  which  we  adapted  a  Goal  Fabric 
Model,  described  in  Chapter  4  of  this  report,  for  linking  current  issues  to  desired  re¬ 
sults  and  actions,  objectives,  and  goals.  This  approach  resulted  in  numerous  recom¬ 
mendations,  which  we  then  grouped  into  ten  intervention  categories.  Later,  we  em¬ 
ployed  a  top-down,  purpose-driven  approach  in  which  we  formulated  a  statement  of 
purpose  and  a  set  of  desired  outcomes  for  the  DES.  This  approach  resulted  in  two 
major  categories  of  recommendations:  changes  to  training  and  deployment  of  a 
management  information  system. 

ORGANIZATION  OF  THIS  REPORT 

This  chapter  introduced  readers  to  the  DES,  the  primary  questions  addressed  in  our 
research,  and  our  study  methodology. 

Chapter  2  presents  some  background  on  this  study.  That  chapter  describes  organiza¬ 
tional  responsibility  for  training  in  the  DES  and  highlights  findings  and  recommen¬ 
dations  from  earlier  reports  that  establish  the  context  for  this  project. 

Chapter  3  articulates  a  purpose  statement  and  a  set  of  desired  outcomes  for  the  DES 
and  suggests  using  these  constructs  as  the  basis  for  assessing  system  performance. 
(The  system  purpose  statement  serves  as  the  touchstone  for  recommendations  pre¬ 
sented  in  Chapters  5  and  6).  Chapter  3  also  identifies  the  DES  external  customers  and 
describes  the  organizational  context  in  which  the  DES  operates.  That  chapter  de¬ 
scribes  the  system  operating  framework  common  to  all  the  military  departments  as 
well  as  numerous  cases  of  variability. 

Chapter  4  describes  the  issues  identified  by  the  primary  participants  in  the  course  of 
our  issues-driven  approach.  That  chapter  presents  our  analysis  of  how  those  issues 
translate  into  desired  results,  actions,  objectives,  and  goals  for  achieving  more  con¬ 
sistent  application  of  disability  policy.  Chapter  4  also  presents  numerous  recom¬ 
mendations  for  specific  interventions,  grouped  into  ten  categories;  the  interventions 
in  two  of  these  categories— training  and  development  of  a  management  information 
system— are  particularly  significant  and  are  developed  further  in  Chapters  5  and  6. 

Chapter  5  presents  an  extensive  training  needs  analysis  together  with  an  analysis  of 
other  considerations  in  training  design.  It  concludes  with  a  detailed  discussion  of 
recommendations  for  a  training  intervention  program. 

Chapter  6  describes  the  structure  of  a  management  information  system  for  monitor- 
ing  performance  of  the  DES  (at  the  DoD,  military  department,  and  military  treatment 
facility  [MTF]  levels),  which  is  necessary  to  assess  training  effectiveness. 

Chapter  7  presents  a  cost-benefit  analysis  of  the  training  intervention  recommended 
in  Chapter  5. 

Chapter  8  offers  our  conclusions  and  observations.  In  particular,  we  discuss  the  value 
of  purposefully  establishing  a  system  performance  perspective  with  which  to  address 
other  complex  issues  beyond  the  scope  of  this  report. 


Chapter  Two 

BACKGROUND  ON  THE  DISABILITY  EVALUATION 

SYSTEM 


Department  of  Defense  policy,  as  reflected  in  DoD  Directive  1332.18  and  DoD 
Instruction  1332.38,  asserts  that  the  Disability  Evaluation  System  is  established  to 
conduct  physical  disability  evaluation  in  a  consistent  and  timely  manner.  In  light  of 
this  stated  policy,  this  chapter  presents  background  information  on  training  in  the 
DES  and  recommendations  from  previous  studies  of  the  system. 

TRAINING  AS  A  CENTRAL  FOCUS  OF  THIS  STUDY 

DoD  Directive  1332.18  (1996)  emphasizes  the  role  of  training  in  achieving  the  DES 
goal  of  conducting  physical  disability  evaluation  in  a  consistent  and  timely  manner. 
It  directs  the  Assistant  Secretary  of  Defense  for  Force  Management  Policy 
(ASD/FMP),  in  coordination  with  the  Assistant  Secretary  of  Defense  for  Health  Affairs 
(ASD/HA)  and  the  Assistant  Secretary  of  Defense  for  Reserve  Affairs  (ASD/RA),  to 
develop  and  maintain  a  program  of  instruction  for  the  DES. 

DoD  Directive  1332.18  also  directs  the  ASD/HA  to  develop  and  maintain  a  program 
of  instruction  for  use  by  MTFs  on  the  preparation  of  medical  evaluation  boards^  for 
physical  disability  cases,  and  a  program  of  instruction  for  use  by  PER  adjudicators 
and  appellate  review  authorities  on  the  medical  aspects  of  physical  disability 
adjudication,  to  include  the  application  of  the  Veterans  Administration  Schedule  for 
Rating  Disabilities  (VASRD).  Appendbc  A  of  this  report  describes  the  organizational 
responsibilities  for  the  DES  within  the  Office  of  the  Under  Secretary  of  Defense 
(Personnel  and  Readiness)  and  the  military  departments. 

In  sponsoring  this  research,  the  Principal  Deputy  ASD/FMP  also  emphasized  the  role 
of  training,  one  of  many  management  tools  for  developing  and  sustaining  the 
knowledge,  skills,  and  behaviors  needed  to  produce  the  desired  results  of  the  DES  in 
a  consistent  manner.  In  addition,  by  uniformly  interpreting  and  communicating 
disability  policy,  training  can  play  a  key  role  in  conveying  a  “consistent  policy 


^To  avoid  confusion  with  the  use  of  the  acronym  MEB— which  is  commonly  used  to  indicate  the  group  of 
physicians  who  convene  as  a  board,  the  narrative  summary,  and  the  complete  disability  case  file — we  spell 
out  “Medical  Evaluation  Board”  when  referring  to  the  group  of  physicians  who  convene  as  a  board 
(including  those  who  pass  records  among  themselves  without  actually  convening  a  board).  We  refer  to  the 
narrative  summary  as  the  “narrative  summary”  and  the  disability  case  file  as  the  “medical  board.” 
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application”  message.  Consistent  policy  application  is  more  likely  to  occur  when 
each  primary  participant  is  able  to  apply  the  body  of  knowledge  and  skills  learned  in 
training  that  are  targeted  to  produce  the  desired  results  on  a  specific  job.^ 

The  focus  on  training  within  the  DES  is  not  new;  it  dates  back  at  least  to  the  begin¬ 
ning  of  the  1990s.  A  1992  DoD  Inspector  General  Audit  Report  recommended  a  joint 
training  program  for  disability  evaluators  that  includes,  as  a  minimum,  line-of-duty 
(LOD)  criteria,  presumption-of-fitness  determinations,  prior-to-service  criteria,  ap¬ 
plication  of  the  VASRD,  and  preparation  of  documentation  to  support  PER  decisions. 
The  ASD/HA  formally  replied  on  February  13,  1992,  that  the  Office  of  the  ASD/HA 
would  develop  a  joint  training  program  for  newly  assigned  members  of  the  boards 
composing  the  DoD  DES  to  promote  consistency  in  the  application  of  the  disability 
separation  laws.^  As  of  this  writing,  however,  the  responsibilities  outlined  in  DoD 
Directive  1332.18  notwithstanding,  no  such  joint  program  has  been  developed. 

Even  in  the  absence  of  training  standards  prescribed  by  the  Office  of  the  Secretary  of 
Defense  (OSD)  or  a  program  of  instruction,  the  military  departments  conduct  train¬ 
ing  for  primary  participants.  The  Army  Physical  Disability  Agency  develops  and 
delivers  a  training  program  tailored  to  the  needs  of  Army  adjudicators  and  invites  a 
limited  number  of  adjudicators  from  the  other  military  departments  to  attend.  Other 
Army  primary  participants  aggressively  and  continuously  seek  to  attend  the  course  as 
reflected  in  data  from  the  December  1999  Army  Adjudicators’  Training  Course, 
which  showed  that  only  two  of  the  68  attendees  represented  the  target  training 
population  of  practicing  adjudicators. 

The  Army  Physical  Disability  Agency  and  the  Air  Force  and  Department  of  the  Navy 
Physical  Evaluation  Boards  (all  three  groups  reside  within  the  personnel  community) 
each  organize  and  conduct  conferences  or  seminars  designed  primarily  for  their  own 
PEBLOs.  Because  demand  for  disability  training  is  high,  other  primary  participants  of 
the  DES  also  attend  on  a  space-available  basis,  including  a  limited  number  of  partic¬ 
ipants  from  the  other  military  departments. 

The  following  section  establishes  the  context  within  which  this  study  took  place, 
starting  with  the  findings  of  the  1992  DoD  Inspector  General  Audit  Report. 


2 

As  noted  in  Chapter  1,  this  study  originally  focused  on  training  interventions  to  achieve  more-consistent 
application  of  disability  policy  across  and  within  military  departments.  As  the  study  progressed,  it  became 
clear  that  consistent  policy  application  is  simply  one  means  of  improving  overall  system  performance,  and 
It  alone  is  not  sufficient  for  developing  comprehensive  training  recommendations. 

^The  full  text  of  the  ASD/HA  response  to  the  report  of  the  Inspector  General  is  as  follows:  ‘‘By  1  March 
1992.  the  ASD(HA)  will  forward  a  memorandum  to  the  Secretaries  of  the  Militar>^  Departments  and  the 
President.  Uniformed  Services  University  of  the  Health  Sciences  (USUHS)  requesting  the  appointment  of 
2  members  from  each  Service  and  from  USUHS  to  establish  a  working  group  whose  responsibility  it  will  be 
to  identify  the  subject  matter  for  the  joint  training  program.  The  working  group  will  be  established  by  1 
April  1992.  The  Center  for  Interactive  Media  at  the  USUHS  will  develop  an  interactive  video  from  the 
subject  matter  input  from  the  working  group.  Completion  of  the  training  program  will  require  an 
estimated  12  months.  Estimated  date  of  completion  of  the  joint  training  program  is  I  June  1993,” 
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FINDINGS  AND  RECOMMENDATIONS  FROM  PREVIOUS  STUDIES  OF  THE 
DISABILITY  EVALUATION  SYSTEM 

This  section  summarizes  findings  and  recommendations  from  related  recent  reports 
that  made  recommendations  similar  to  the  ones  in  this  report.  The  first  two  sum¬ 
maries  in  this  section  discuss  the  performance  of  the  DoD  Disability  Evaluation 
System.  The  next  two  summaries  discuss  the  performance  of  the  Army  Physical 
Disability  Agency,  and  the  final  summary  in  this  section  discusses  the  performance 
of  the  veterans  disability  compensation  programs  in  the  Department  of  Veterans 
Affairs  (VA). 

Medical  Disability  Discharge  Procedures  (June  8, 1992) 

The  DoD  Inspector  General  audited  the  military  departments’  medical  disability  dis¬ 
charge  procedures  (Office  of  the  Inspector  General,  1992)  to  determine  whether  ser¬ 
vice  members  identified  as  medically  disabled  were  expeditiously  discharged  from 
the  military  departments,  and  to  assess  the  effectiveness  of  the  discharge  process  and 
related  internal  controls.  The  Office  of  the  Inspector  General  determined  that  the 
DoD  disability  system  was  neither  efficient  nor  economical.  Disability  cases  were  not 
processed  promptly,  and  service  members  were  incorrectly  rated  for  their 
disabilities.  The  Office  of  the  Inspector  General  found  that  OSD  guidance  was 
inadequate  and  the  DoD  lacked  adequate  oversight  of  the  disability  process. 

•  In  particular,  the  report  noted  among  other  findings  that  the  DoD  Directive 
1332.18 — originally  titled  Separation  from  Military  Service  by  Reason  of  Physical 
Disability,  February  25,  1986— did  not  contain  standard  time  frames  for 
processing  medical  boards  and  evaluation  boards  and  did  not  provide  adequate 
criteria  for  rating  disabilities  and  prior-to-service  conditions.  It  also  found  that 
military  personnel  were  assigned  to  serve  on  PEBs  without  any  training  on  how 
to  evaluate  disability  cases. 

•  High  turnover  among  board  members  plus  the  lack  of  formal  training  con¬ 
tributed  to  rating  deficiencies. 

•  Frequently,  medical  boards  did  not  contain  an  LOD  determination  or  the  state¬ 
ment  was  inadequate. 

•  A  system  did  not  exist  to  collect  data  from  the  PEBs  on  how  quickly  cases  were 
processed  or  the  number  of  cases  for  each  medical  condition. 

The  DoD  Directive  1332.18  was  revised  and  re-titled  as  Separation  or  Retirement  for 
Physical  Disability,  and  was  reissued  on  November  4,  1996.  The  supplemental  new 
DoD  Instruction  1332.38,  Physical  Disability  Evaluation,  November  14,  1996,  speci¬ 
fies  standard  processing  times,  as  follows,  and  includes  criteria  for  rating  disabilities 
and  prior-to-service  conditions; 

Not  more  than  30  days  from  the  date  the  physician  dictates  the  Medical  Evaluation 
Board  Report  to  the  date  the  Physical  Evaluation  Board  receives  the  medical  board. 
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Not  rnore  than  90  calendar  days  to  conduct  a  Medical  Evaluation  Board  or  physical 
examination  in  cases  of  Reserve  component  members  referred  solely  for  a  fitness  de¬ 
termination  of  a  non-duty-related  condition. 

Not  more  than  40  days  from  date  the  Physical  Evaluation  Board  receives  the  medical 
board  or  physical  examination  report  to  the  date  of  the  determination  of  the  final  re¬ 
viewing  authority  as  prescribed  by  the  Secretary  of  the  military  department. 

Based  on  our  interviews  with  the  primary  participants  and  our  other  observations, 
individuals  still  to  this  day  are  assigned  to  serve  on  PEBs  usually  without  the  benefit 
of  standardized  training  on  how  to  evaluate  disability  cases. 

High  turnover  among  PEB  members  and  the  PEB  approving  authorities  still  exists 
and  formal  training  is  still  offered  only  infrequently.  Primary  participants  from  all  of 
the  military  departments  reported  that  now,  as  in  1992,  medical  boards  frequently  do 
not  contain  an  LOD  determination  or  the  statement  is  inadequate.  By  January  1992, 
the  Department  of  the  Navy  Disability  Evaluation  System  received  funding  for  a 
Management  Information  System  (Office  of  the  Assistant  Secretary  of  Defense, 
Health  Affairs,  1992),  the  genesis  of  the  current  Joint  Disability  Evaluation  Tracking 
System. 


Preliminary  Functional  Economic  Analysis  (November  1993) 

To  address  the  findings  of  the  DoD  Inspector  General  Audit  Report  (1992),  the  Office 
of  the  ASD/HA  convened  the  Joint  Service  Disability  Working  Group  to  analyze  and 
recommend  improvements  for  the  disability  evaluation  process  using  the  Corporate 
Information  Management  (CIM)  methodology^  (Joint  Service  Disability  Working 
Group  and  Systems  Research  and  Applications  Corporation,  1993).  The  work  group 
determined  that  although  the  medical  and  disability  investigation  functions  were 
similar  among  the  military  departments  and  compliant  with  the  law,  the 
implementation  of  the  disability  evaluation  process  differed  among  the  military  de¬ 
partments.  Of  the  differences,  the  work  group  reported  that  only  one  was  justified 
and  should  be  preserved;  the  difference  in  fitness  and  retention  standards  among  the 
services  due  to  mission  requirements. 

The  work  group  noted  that  the  future  DES  should  be  performance-based  and  rec¬ 
ommended  the  following  actions  to  achieve  the  desired  performance-based  DES  of 
the  future: 

•  The  Office  of  the  Secretary  of  Defense  and  military  departments  should 

—  issue  specific  policy  guidance 

—  cooperate  with  the  VA 

—  employ  sound  business  practices. 


^The  CIM  methodology  rigidly  focuses  on  a  functional  process  improvement  cycle  that  includes:  defining 
objectives,  strategies,  and  a  baseline;  analyzing  functional  processes:  evaluating  alternatives;  planning  for 
implementation;  approving  proposed  changes;  executing  new  processes  and  systems;  and  comparing  the 
results  with  the  first  stage  in  the  cycle,  defining  objectives,  and  other  steps  in  the  process. 
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•  Disability  Evaluation  System  leaders  monitor  system  performance,  learn  from  it, 
and  take  action  based  on  the  performance  measures. 

The  work  group  developed  two  alternatives  to  produce  a  uniform  process  across  the 
military  departments  and  timely  fitness  and  disability  determinations: 

1.  Alternative  A  recommended  basic  improvements  and  actions  that  would  simplify 
the  process,  such  as 

•  transfer  policymaking  to  the  Office  of  the  Assistant  Secretary  of  Defense  (Person¬ 
nel  and  Readiness) 

•  create  a  Disability  Evaluation  Council 

•  develop  an  automated  information  system  with  monitoring  and  reporting  ca¬ 
pability 

•  develop  and  field  education  programs. 

2.  Alternative  B,  which  presupposed  Alternative  A  implementation,  included  signifi¬ 
cant  functional  changes  such  as  eliminating  informal  reviews  not  required  by  law; 
moving  fitness  and  retention  decisions  to  local  (base/post)  level;  and  later  moving 
disability  rating  decisions  to  the  local  level  once  an  automated  disability  rating 
“advisor”  could  be  fielded  at  the  local  level. 

Since  the  Preliminary  Functional  Economic  Analysis  report  was  published,  policy¬ 
making  has  been  transferred  to  the  Office  of  the  Under  Secretary  of  Defense 
(Personnel  and  Readiness)  and  the  Disability  Advisory  Council  was  established. 
Establishing  the  Joint  Disability  Evaluation  Tracking  System  can  be  seen  as  an  initial 
effort  to  create  an  automated  information  system  capable  of  both  monitoring  and 
reporting. 

Disability  Payments  to  Military  Personnel  (December  1989) 

In  1988  to  1989,  the  U.S.  Army  Audit  Agency  (USAAA)  conducted  an  audit  of  disability 
payments  to  military  personnel  (U.S.  Army  Audit  Agency,  1989).  Although  the  USAAA 
reported  that  the  PEBs  properly  and  consistently  adjudicated  cases,  it  also  reported 
problems  with  case  processing  time  and  the  ability  of  the  Army  Physical  Disability 
Agency  to  measure  system  performance  so  that  managers  could  identify  and  correct 
unnecessary  delays.  The  USAAA  estimated  that  delays  in  case  processing  cost  the 
Army  about  $19.4  million  in  active  duty  personnel  costs  annually. 

The  USAAA  recommended  that  MTFs  properly  prepare  all  reports  and  forms  and 
expedite  processing  of  medical  boards.  It  specifically  recommended  that  the  Army 
Physical  Disability  Agency 

•  coordinate  with  the  Office  of  the  Surgeon  General  and  the  Total  Army  Personnel 
Command  to  develop  processing  time  standards  to  cover  all  key  segments  in  the 
DES 

•  develop  a  standard  system  for  recording  and  reporting  action  dates  for  each  key 
segment  of  the  DES 
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•  periodically  prepare  reports  based  on  data  in  the  information  system  and  dis¬ 
seminate  the  reports  to  responsible  activities.^ 

Follow-up  Audit  of  Disability  Payments  to  Military  Personnel  (December 
1994) 

In  its  follow-up  audit  of  the  disability  payments  to  military  personnel,  the  USAAA 
found  that  problems  still  existed  in  the  DBS  (U.S.  Army  Audit  Agency,  1994).  The 
USAAA  reported  that  the  Army  Physical  Disability  Agency  needed  to  improve  its 
oversight  of  disability  case  processing,  clearly  define  responsibilities  for  enforcing 
requirements  of  the  DBS,  and  improve  its  management  information  systems.  It 
specifically  recommended  that  the  Army  Physical  Disability  Agency 

•  update  Army  Regulation  635-40  (AR,  1990)  to  clearly  delineate  time  standards  for 
each  segment  of  the  disability  process  and  requirements  for  activities  to  monitor 
the  timeliness  of  case  processing 

•  establish  quality  assurance  controls  and  periodically  verify  the  accuracy  of  au¬ 
tomated  information. 

To  date.  Army  Regulation  635-40  has  not  been  revised. 

The  USAAA  did  note  that  the  Army  Physical  Disability  Agency  was  taking  a  number  of 
actions  to  improve  its  management  of  the  disability  process  including  participating 
in  a  DoD  study  group  to  review  the  disability  process  with  all  military  departments; 
conducting,  along  with  the  Office  of  the  Surgeon  General,  an  analysis  of  the  Army’s 
disability  evaluation  system;  making  arrangements  for  MTFs  to  obtain  automation 
equipment;  and  reviewing  its  organization  structure  for  more  cost-effective  business 
approaches. 

The  USAAA  also  reported  that  processing  time  was  still  too  high.  Although  the  Army 
Physical  Disability  Agency  had  developed  time  standards  and  implemented  a  mea¬ 
surement  system  for  case  processing  time,  actual  case  processing  times  did  not  de¬ 
crease.  The  USAAA  noted  that  the  increased  number  of  disability  cases  due  to 
Operation  Desert  Shield  and  Operation  Desert  Storm  may  have  contributed  to  time¬ 
liness  problem. 

The  USAAA  recommended  that  the  Deputy  Chief  of  Staff  for  Personnel 

•  coordinate  with  the  Surgeon  General  to  develop  a  plan  to  reduce  processing  time 

•  require  the  Army  Physical  Disability  Agency  to  periodically  report  case  process¬ 
ing  time 

•  monitor  case  processing  performance. 

In  addition,  the  USAAA  advocated  that  the  Surgeon  General 

^Activities,  as  used  here,  is  a  department  term  referring  to  organizations  or  units 
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•  require  that  major  medical  commands  include  the  timeliness  of  disability  case 
processing  in  command  personnel  reviews  and  inspections 

•  monitor  processing  time 

•  require  doctors  to  provide  reasons  for  delays 

•  ensure  that  administrative  staff  members  possess  proper  knowledge  on 
disabilities. 

The  Veterans’  Claims  Adjudication  Commission  Report  to  Congress 
Pursuant  to  Public  Law  103-446  (December  1996) 

Section  402(e)  (2)  of  Public  Law  103-446  called  for  the  establishment  of  the  Veterans’ 
Claims  Adjudication  Commission,  which  examined  the  performance  of  disability 
compensation  programs  within  the  Department  of  Veterans  Affairs.  The  Commission 
concluded  that,  in  regard  to  the  adjudicative  and  appellate  process  and  procedures, 
“the  shortcomings  of  the  existing  system  are  many  and  varied.” 

Lack  of  a  clear  statement  of  purpose  for  veterans’  disability  compensation  is  a  huge 
shortcoming  that  makes  it  difficult  to  both  determine  performance  measures  and  ac¬ 
tually  measure  performance.  Although  language  in  Title  38,  U.S.C.,  and  language  in 
VA  regulations  strongly  suggest  that  disability  compensation  is  intended  to  com¬ 
pensate  for  lost  earning  capacity,  and  numerous  congressional  committee  reports 
express  similar  intent,  no  clear  statement  of  purpose  exists  in  statute. 

The  report  (GPO,  1996)  makes  numerous  recommendations;  however,  it  emphasizes 
that  no  single  intervention  is  likely  to  impact  system  performance  sufficiently  to  alter 
“perceptions  that  the  VA  system  is  failing,  is  not  efficient,  and/or  does  not  provide 
appropriate  service  to  veterans.”  The  report  laments  that  credible  data  and  long¬ 
term  analysis  of  program  trends  do  not  support  decisionmaking  in  the  VA.  It 
concludes  that,  among  other  things,  the  Congress  and  Department  of  Veterans 
Affairs  require  objective  and  contextual  information  to  inform  attempts  to  redesign 
or  improve  the  VA  disability  compensation  system.  Among  the  many 
recommendations  specified  by  the  commission,  the  follovnng  relate  to  this  report: 

•  Congress  should  amend  Title  38,  U.S.C.,  to  clearly  state  the  purpose  of  the 
veterans’  pension  program. 

•  The  Secretary  of  Veterans  Affairs  should  actively  support  and  encourage  the  ef¬ 
forts  of  the  VA’s  Chief  Information  Officer  to  execute  the  Chief  Information  re¬ 
sponsibilities  and  authorities  (that  is,  establish  a  management  information  sys¬ 
tem  capable  of  monitoring  performance  and  establishing  long-term  trends). 

•  Routine  analysis  of  operations  should  be  based  on  a  single  set  of  predetermined 
performance  measures. 

•  To  enhance  accountability,  the  Veterans’  Benefits  Administration  and  the  Board 
of  Veterans’  Appeals  should  incorporate  organizational  goals  and  objectives  (at 
the  department,  administration,  and  board  levels)  into  individual  performance 
plans. 
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•  The  Veterans’  Benefits  Administration  and  Board  of  Veterans’  Appeals  should 
integrate  timeliness  of  processing  into  their  Quality  Control  and  Quality 
Assurance  frameworks. 

The  next  chapter  of  this  report  suggests  a  statement  of  purpose  and  a  set  of  desired 
outcomes  for  the  DoD  DBS.  It  explains  the  organizational  setting  of  the  DBS  and  de¬ 
scribes  the  common  DBS  operating  system  across  military  departments  as  well  as 
numerous  cases  of  variability. 


Chapter  Three 

DISABILITY  EVALUATION  SYSTEM  OVERVIEW 


This  chapter  describes  the  DoD  Disability  Evaluation  System  from  five  perspectives: 
(1)  its  stated  purpose  and  desired  outcomes,  (2)  its  external  customers,  (3)  its  organi¬ 
zational  setting,  (4)  the  DES  operating  framervork  that  exists  in  common  across  the 
military  departments,  and  (5)  aspects  of  DES  operations  that  are  unique  to  certain 
military  departments.  Examining  the  DES  from  these  perspectives  is  important  to 
gaining  a  thorough  understanding  of  both  the  system  and  the  recommendations  de¬ 
scribed  in  this  report. 

The  first  section  of  this  chapter  discusses  the  purpose  of  the  DES.  We  give  the  sys¬ 
tem’s  purpose  the  preeminent  position  in  this  chapter  because  it  is  central  to  any 
effort  to  improve  the  operation  of  the  DES.  The  first  section  also  outlines  a  set  of  de¬ 
sired  outcomes  that  explicitly  states  the  intended  results  of  operating  the  system  to 
achieve  its  purpose.  The  second  section  of  this  chapter  identifies  the  DES  customers 
and  their  expectations  of  the  system. 

Because  the  DES  exists  within  an  organizational  setting,  the  third  section  of  this 
chapter  describes  the  elements  of  the  system  within  that  setting,  the  organizational 
location  of  those  elements  within  the  military  departments,  and  the  relationships 
and  flow  of  information  among  the  organizations  that  operate  the  various  elements 
of  the  DES  and  among  other  organizations  inside  and  outside  the  military  depart¬ 
ments. 

The  final  section  of  this  chapter  summarizes  the  common  operating  framework  of 
the  DES  across  the  military  departments  and  aspects  of  DES  operations  that  are 
unique  to  certain  military  departments,  in  other  words,  instances  of  variability. 


PURPOSE  STATEMENT  AND  DESIRED  OUTCOMES 

A  purpose  statement  describes  the  fundamental  and  unchanging  reason  for  the 
DBS’s  existence.  It  differentiates  the  DES  from  other  human  resource  management 
systems  (and  tools)  and  is  the  foundation  for  designing,  redesigning,  organizing,  and 
monitoring  every  aspect  of  the  system. 

Lacking  explicit  direction  from  the  Office  of  the  Secretary  of  Defense  (OSD)  on  the 
purpose  of  the  Disability  Evaluation  System,  the  military  departments  tend  to 
interpret  DoD  policy  language  to  fill  the  directional  void  and  operate  their  systems 
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accordingly.  Their  statements  of  purpose  (or  "objectives”  or  “mission  statement”) 
include  various  renditions  and  combinations  of  five  themes: 

1.  Maintain  a  fit  force. 

2.  Provide  compensation  and  benefits. 

3.  Remove  unfit  members  from  active  duty. 

4.  Balance  the  interests  of  the  government  and  the  service  member. 

5.  Serve  both  active  and  Reserve  service  members  implicitly;  the  Department  of  the 
Navy  explicitly  includes  the  Reserve  components. 

The  military  departments’  various  DES  purpose  statements,  along  with  the  mission 
statement  set  forth  by  the  Joint  Service  Disability  Working  Group  in  1993,  appear  in 
Appendix  B. 

The  system’s  stated  purpose  should  be  the  foundation  for  any  major  change  in  the 
direction,  structure,  or  operation  of  the  DES.  After  extensive  discussions  with  its  pri¬ 
mary  participants  and  based  on  a  review  of  OSD  and  military  department  docu¬ 
ments,  we  formulated  the  following  statement  of  purpose  of  the  Disability  Evaluation 
System: 

The  Disability  Evaluation  System  exists  to  evaluate  service  members  with  potentially 
unfitting  conditions^  in  a  fair,  consistent,  efficient,  and  timely  manner  and,  likewise, 
to  remove  those  unable  to  fulfill  the  duties  of  their  office,  grade,  rank,  or  rating,  and 
determine  a  disability  rating  percentage  for  those  removed. 

This  proposed  statement  of  purpose  explains  why  the  system  exists.  And  similarly, 
the  desired  outcomes  explicitly  describe  the  intended  results  of  operating  the  system 
to  achieve  its  stated  purpose:  results  that  matter  to  DES  customers.  As  was  the  case 
with  the  lack  of  a  common  purpose  statement,  we  found  no  shared  understanding 
among  the  military  departments  or  between  the  OSD  and  the  military  departments 
regarding  a  set  of  desired  outcomes.^  As  a  result,  in  addition  to  proposing  a  purpose 
statement,  we  also  propose  the  following  set  of  desired  system  outcomes,  which  if 
met,  will  lead  to  achieving  that  purpose:^ 


"Unfitting  condition”  is  a  term  commonly  used  in  DES  policy.  Although  not  explicitly  defined  in  DoD  or 
U.S.  statutory  documents,  the  term  could  be  said  to  refer  to  “a  medical  condition  resulting  from  disease  or 
injury  that  makes  a  service  member  unfit  to  perform  the  duties  of  the  member’s  office,  grade,  rank,  or 
rating”  (DoD  Directive  1332.18, 1996,  p.  2;  DoD  Directive  6130.3, 1994,  p.  1). 

2 

The  OSD  relies  on  time  standards  for  processing  medical  boards  through  the  Disability  Evaluation 
System  to  assess  system  performance.  Chapter  6  outlines  a  comprehensive  performance  measurement 
system  that  relates  output  measures,  such  as  medical  board  processing  time,  to  desired  outcomes. 

q 

We  derived  the  purpose  and  outcomes  from  the  DoD  and  military  department  documents  and  from 
interviews  with  primary  participants.  As  a  result,  they  should  be  generally  acceptable  to  decisionmakers  in 
the  OSD  and  military  departments;  however,  we  did  not  attempt  to  secure  agreement  from  those 
decisionmakers.  Rather,  we  believe  it  is  essential  for  the  ASD/FMP,  in  consultation  with  the  ASD/RA  and 
ASD/HA,  to  decide  on  a  stated  DES  purpose  and  desired  outcomes,  using  our  proposed  framework  as  a 
starting  point.  The  objective  of  their  deliberations  is  a  common  framework  to  develop  a  sense  of 
ownership  of  the  DES  purpose  and  desired  outcomes~the  purpose  and  desired  outcomes  inform  all  other 
decisions  and  interventions.  This  purpose-driven  approach  is  discussed  in  greater  detail  in  Appendix  F. 
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1.  Service  members  having  a  similar  condition  and  similar  office,  grade,  rank,  or 
rating  receive  similar  fitness  decisions  within  the  military  department. 

2.  Service  members  found  unfit  receive  similar  disability  ratings  for  similar  condi¬ 
tions  across  and  within  the  military  departments. 

3.  Service  members  freely  and  appropriately  exercise  their  rights  to  administrative 
due  process. 

4.  Service  members  return  to  duty,  separate,  or  retire  for  disability  in  a  timely  man¬ 
ner. 

5.  Primary  participants  perform  their  duties  as  efficiently  as  possible  so  that,  collec¬ 
tively,  they  return  service  members  to  duty,  or  separate  or  retire  service  members 
for  disability  in  a  fair,  consistent,  and  timely  manner. 

The  two  major  DBS  interventions  recommended  in  Chapters  5  and  6  flow  from  the 
proposed  purpose  statement  and  set  of  desired  outcomes.  Because  a  shared  purpose 
and  shared  understemding  of  desired  outcomes  do  not  currently  exist,  reaching  con¬ 
sensus  on  a  purpose  statement  and  a  set  of  desired  outcomes  is  an  important  first 
step  in  the  development  of  interventions  to  improve  the  performance  of  the  DBS.  We 
recommend  that  the  OSD  make  reaching  this  consensus  a  key  initial  action. 

We  used  the  purpose  statement  and  outcomes  proposed  here  to  develop  the  two 
major  recommendations  discussed  in  Chapters  5  and  6,  recognizing  that  some  of  the 
specifics  will  change  if  the  OSD  modifies  the  proposed  purpose — the  foundation  of 
the  DBS’s  existence.  However,  based  on  our  discussions  with  the  primary  partici¬ 
pants,  we  believe  that  the  purpose  statement  and  outcomes  we  suggest  are  close  to 
those  that  will  eventually  be  agreed  upon. 

EXTERNAL  CUSTOMERS 

The  military  departments  operate  their  DBSs  to  benefit  two  customers:  individual 
service  members  and  individual  military  services.  The  system’s  stated  purpose  and 
desired  outcomes  define  customer  expectations.  As  such,  service  member  expecta¬ 
tions  center  on  similar  dispositions  (among  service  members  in  similar  circum¬ 
stances)  and  on  due  process.  Service  expectations  center  on  expeditious  processing 
and  efficient  operations. 

For  a  discussion  of  measuring  system  performance  in  terms  of  how  well  it  meets  ex¬ 
ternal  customer  expectations,  see  Chapter  6. 

THE  ORGANIZATIONAL  SETTING 

The  DBS  operates  to  achieve  its  stated  purpose  within  a  larger  organizational  con¬ 
text.  According  to  DoD  policy  documents,  the  DBS  is  composed  of  four  major  ele¬ 
ments.  Significantly,  no  single  organization  within  the  military  departments  “owns” 
all  of  the  elements  of  the  system.  The  DBS  interfaces  with  other  systems  within  the 
military  departments.  The  organizations  that  constitute  the  DBS  exchange  opera¬ 
tionally  critical  information  with  other  organizations  within  the  military  departments 
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and  with  one  external  organization,  the  Department  of  Veterans  Affairs.  The  DES’s 
operation  within  this  context  directly  affects  its  performance. 

The  Elements  of  the  Disability  Evaluation  System 

DoD  policy  documents  identify  the  four  elements  that  constitute  the  Disability 
Evaluation  System: 

•  medical  evaluation 

•  physical  disability  evaluation,  to  include  appellate  review 

•  counseling 

•  final  disposition. 

A  service  member  enters  the  DES  when  a  medical  evaluation  calls  into  question  his 
or  her  ability  to  meet  medical  retention  standards  to  perform  military  duties.^  A 
member  who  does  not  meet  medical  retention  standards  progresses  to  a  physical 
disability  evaluation. 

Primarily  while  undergoing  medical  and  physical  disability  evaluations,  the  service 
member  receives  counseling  regarding  what  to  expect  throughout  all  phases  of  the 
disability  evaluation  process,  the  significance  and  consequences  of  the  determina¬ 
tions  that  are  made,  and  his  or  her  rights,  benefits,  and  entitlements. 

A  member  who  disagrees  with  the  physical  disability  evaluation  findings  and  rec¬ 
ommendations  may  redress  that  disagreement  through  appellate  review. 

Appropriate  personnel  authorities  accomplish  final  disposition  of  the  service  mem¬ 
ber’s  case  by  issuing  orders  and  instructions  to  implement  the  determination  of  the 
respective  military  department’s  final  reviewing  authority.  The  service  member  exits 
the  DES  by  returning  to  duty,  separating  (with  or  without  compensation),  or  retiring 
for  disability  or  length  of  service.  Figure  3.1  illustrates  the  four  elements  that  consti¬ 
tute  the  DES.  It  notes  that  the  counseling  element  primarily  occurs  simultaneously 
with  the  medical  evaluation  element  and  the  physical  disability  evaluation  portion  of 
the  second  element. 

The  admittedly  simple  representation  of  the  DES  shown  in  Figure  3.1  belies  its 
underlying  complexity.  Some  of  that  complexity  arises  because  no  single  organi¬ 
zation  owns  all  the  elements  of  the  system. 


This  introduction  to  the  four  elements  of  the  DES  is  based  on  DoD  disability  policy  documents  and  does 
not  include  the  variations  that  exist  across  the  military  departments,  such  as  the  alternative  route  into  the 
Army  DES  through  the  Military  Occupational  Specialty  Medical  Retention  Board.  The  last  section  of  this 
chapter  specifies  the  operational  and  structural  differences  that  we  identified  within  the  three  military 
departments'  DESs  in  the  context  of  the  system  framework  that  is  common  across  the  military 
departments. 
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Figure  3.1— Elements  of  the  Disability  Evaluation  System 


Organizational  Location 

The  Disability  Evaluation  System  is  but  one  of  the  systems  supporting  the  personnel 
(human  resource  management)  communities  of  the  military  departments.  The  sys¬ 
tem,  ideally,  supports  the  broader  goals  of  the  personnel  functions  of  which  it  is  a 
part,  which  in  turn  support  broader  enterprise  goals  of  the  military  departments  and 
the  DoD.  In  fact,  the  “maintain  a  fit  force”  theme  identified  in  the  various  purpose 
statements  of  the  military  departments,  as  noted  earlier  in  this  chapter,  highlights 
the  linkage  between  the  DES  and  the  enterprise  goals  of  the  military  departments 
and  the  DoD  to  maintain  a  fit  force.  As  one  of  many  management  tools  to  support 
the  enterprise  goals,  the  DES  is  the  specific  tool  used  to  evaluate  service  members 
with  medical  conditions  that  make  them  potentially  unfit  to  perform  their  duties, 
and  to  remove  those  service  members  who  are  unable  to  fulfill  the  duties  of  their  of¬ 
fice,  grade,  rank,  or  rating. 

Two  functional  areas  within  the  military  departments  collaborate  to  operate  the  DES: 
the  medical  community  and  the  personnel  community. 

The  Medical  Evaluation  Board,^  which  is  responsible  for  the  medical  evaluation  ele¬ 
ment,  is  organizationally  located  at  the  MTFs.  The  board  is  part  of  the  medical  com¬ 
munity  of  the  military  departments. 


^To  avoid  misunderstanding,  we  avoid  using  the  acronym  MEB,  which  is  commonly  used  to  mean  three 
different  things:  the  group  of  physicians  who  convene  as  a  board,  the  narrative  summary,  and  also  the 
complete  disability  case  file.  Instead,  we  use  the  term  “Medical  Evaluation  Board”  when  referring  to  the 
group  of  physicians  that  convene  as  a  board  (including  those  who  pass  records  among  themselves  without 
actually  convening  a  board),  we  refer  to  the  narrative  summary  as  the  “narrative  summary,”  and  we  refer 
to  the  disability  case  file  as  the  “medical  board.” 
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As  later  chapters  of  this  report  describe,  many  of  the  primary  participants  of  the  DES 
view  the  Physical  Evaluation  Board,  which  is  responsible  for  the  physical  disability 
evaluation  element,  as  the  heart  of  the  system.  The  PEB  is  part  of  the  personnel 
community  in  all  three  military  departments,  although  each  department  positions 
the  PEB  within  a  different  part  of  its  overall  organizational  structure.  The  Army  PEB, 
for  example,  belongs  to  the  Army  Physical  Disability  Agency,  Office  of  the  Adjutant 
General,  Total  Army  Personnel  Command.  The  Department  of  the  Navy  PEB  is  a 
component  of  the  Director,  Naval  Council  of  Personnel  Boards,  Office  of  the 
Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs).  The  Air  Force  PEB 
belongs  to  the  Air  Force  Physical  Disability  Division,  Directorate  of  Personnel 
Program  Management,  Air  Force  Personnel  Center. 

In  addition,  the  higher-level  appellate  review  beyond  the  PEB  in  the  second  element 
of  the  DES  and  the  final  disposition  element  reside  organizationally  within  the  per¬ 
sonnel  community. 

To  achieve  its  purpose,  the  DES  also  interfaces  with  other  organizations  within  the 
military  department:  each  service  member’s  unit  and  the  Office  of  the  Judge 
Advocate  General.  The  service  member’s  unit  has  a  direct  interest  in  the  operation  of 
the  DES  because  the  unit  must  operate  without  a  replacement  for  the  service  mem¬ 
ber  until  final  disposition  of  the  case,  and  the  service  member  receives  legal  advice 
and  representation  from  an  attorney  from  the  Office  of  the  Judge  Advocate  General. 

Figure  3.2  portrays  the  organizational  setting  of  the  DES  within  the  medical  com¬ 
munity  (in  particular,  MTFs)  and  the  personnel  community  and  its  interface  with  the 
other  military  department  organizations:  the  service  member’s  units  and  the  Office 
of  the  Judge  Advocate  General. 

The  flow  of  information  among  these  entities  is  even  more  abundant  than  Figure  3.2 
suggests.  Given  the  organizational  setting  illustrated  in  Figure  3.2,  Figure  3.3  portrays 
the  DES  as  one  of  many  major  systems  that  exchanges  information,  including  one 


Medical 

Community 


RAN  DMA  1228-3.  ^ 


Personnel 

Community 


Figure  3.2— Key  Disability  Evaluation  System  Organizational  Interfaces 
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Figure  3.3— Flow  of  Information  Among  Major  Military  Department  Systems 


system  external  to  the  DoD,  the  Department  of  Veterans  Affairs.  For  the  DBS  to 
operate  smoothly  and  efficiently,  the  military  departments  must  understand,  and 
manage,  these  interfaces  and  information  streams. 

The  military  departments  operate  their  individual  systems  based  on  their  interpreta¬ 
tions  of  DoD  disability  policy.  Careful  examination  revealed  essentially  identical 
system  frameworks,  with  numerous  operational  and  structural  variations,  across  the 
military  departments.  The  following  section  describes  the  system  framework  that  is 
common  across  the  military  departments  in  terms  of  process,  primary  participants 
and  their  roles,  information  examined,  and  range  of  disposition  options  in  each 
phase  of  the  process.  Within  the  context  of  a  common  system  framework,  the  follow¬ 
ing  section  also  specifies  the  operational  and  structural  variations  that  we  identified 
within  the  three  military  departments’  DESs. 

SYSTEM  OPERATING  FRAMEWORK  ACROSS  DEPARTMENTS;  OVERVIEW 
OF  THE  FOUR  DISABILITY  EVALUATION  PHASES 

As  noted  earlier,  DoD  disability  policy  documents  set  forth  policy  within  a  framework 
that  consists  of  four  elements:  (1)  medical  evaluation;  (2)  physical  disability  evalua¬ 
tion,  to  include  appellate  review;  (3)  counseling;  and  (4)  final  disposition  by  the  ap¬ 
propriate  personnel  authorities. 

This  section  describes  the  system’s  operational  framework.  It  reorients  the  policy 
focus  from  “elements”  to  “phases”  through  which  an  individual  service  member’s 
case  may  move,  introducing  a  systems  perspective  of  the  DBS. 
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A  service  member  who  enters  the  DES  can  potentially  pass  through  four  progressive 
phases: 

1.  Medical  evaluation  and  disposition  by  the  Medical  Evaluation  Board 

2.  Physical  disability  evaluation,  including  the  possibility  of  a  formal  hearing,  and 
disposition  by  the  PEB 

3.  Two  or  three  higher  levels  of  appellate  review  beyond  the  PEB 

4.  Final  disposition  by  the  appropriate  personnel  authorities. 

Counseling,  cited  as  the  third  "element”  within  disability  policy  documents,  is  not  a 
separate  phase.  Counseling  aids  service  members  as  they  progress  through  the  first 
two  phases  in  particular.  Figure  3.4  portrays  the  four  phases  of  the  DES  in  relation  to 
the  four  individual  DES  elements. 

When  a  service  member  has  received  maximum  benefit  from  medical  treatment  for  a 
condition  that  may  prevent  the  service  member  from  meeting  medical  retention 
standards,6and  the  service  member  fails  to  improve  or  recover,  he  or  she  may  be  re¬ 
ferred  to  a  Medical  Evaluation  Board  by 

•  a  physician 

•  the  unit  commander,  through  the  MTF  commander 

•  the  service  headquarters  or  higher  command. 

Note:  Other  means  of  referral  to  a  Medical  Evaluation  Board  exist  in  two  military  de¬ 
partments: 

•  The  Army  Military  Occupational  Specialty  Medical  Retention  Board  (MMRB),  an 
administrative  screening  board  that  evaluates  the  ability  of  service  members  with 
a  “permanent  3”  or  "permanent  4”  medical  profile  to  physically  perform  in  a 
worldwide  field  environment  in  their  primary  military  occupational  specialty, 
may  direct  referral  to  a  Medical  Evaluation  Board.  When  the  MMRB  refers  a  ser¬ 
vice  member  to  a  Medical  Evaluation  Board,  the  service  member  must  be  re¬ 
ferred  to  the  PEB,  whether  or  not  the  member  meets  medical  retention 
standards. 

•  The  most  frequent  cause  of  referral  to  the  Medical  Evaluation  Board  in  the 
Department  of  the  Navy  is  that  the  service  member  used  all  of  the  Temporary 
Limited  Duty  available  and  still  requires  medical  treatment.  The  Chief,  Naval 
Operations;  Chief,  Marine  Corps;  Chief,  Naval  Personnel;  and  Chief,  Bureau  of 
Medicine  and  Surgery  may  order  an  MTF  to  convene  a  Medical  Evaluation  Board 
for  a  member  (Secretary  of  the  Navy  Instruction  1850.4D,  1998,  p.  3-2). 


The  seivices  employ  different  medical  retention  standards,  spelled  out  in  U.S.  Department  of  the  Army 
Regulation  40-501  (1995),  and  U.vS.  Department  of  the  Army  Regulation  40-501,  Change  1  (1998);  U.S. 
Secretary  of  the  Navy  Instruction  1850.4D  (1998);  and  U.S.  Department  of  the  Air  Force  Instruction  36- 
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Figure  3.4 — Elements  and  Phases  of  the  DisabOity  Evaluation  System 


Phase  1.  Medical  Evaluation  and  Disposition  by  the  Medical  Evaluation 
Board 

A  service  member's  case  enters  the  Disability  Evaluation  System  when  the  referring 
physician  dictates  a  narrative  summary^  for  a  Medical  Evaluation  Board.® 

Note  the  following  military  department-specific  differences: 

•  Interns  may  write  narrative  summaries  at  the  Army  MTFs. 

•  First-year  residents  generally  write  the  narrative  summaries  at  Department  of  the 
Navy  MTFs. 

•  The  Departments  of  the  Army  and  Navy  refer  service  members  who  have  a  high 
probability  of  nof  returning  to  duty  to  Medical  Evaluation  Boards. 

•  The  Air  Force  refers  service  members  who  have  a  high  probability  of  returning  to 
duty  to  Medical  Evaluation  Boards. 


^Although  practitioners  in  all  the  services  frequently  refer  to  the  narrative  summary  document  as  a 
“medical  board,”  this  report  uses  the  term  medical  board  exclusively  to  refer  to  the  complete  disability 
case  file  including  the  narrative  summary  and  all  other  associated  addenda.  The  Army  calls  the  narrative 
summary  a  “MEB  narrative  summary.” 

®  An  active  or  Reserve  component  service  member  with  a  prognosis  of  death  within  72  hours  and  an  LOD 
Determination  of  Yes  (LOD-Yes)  enters  the  Disability  Evaluation  System  any  time,  day  or  night,  for 
expeditious  disability  retirement  processing,  also  known  as  imminent  death  processing.  Based  on  an 
investigation  of  the  circumstances  surrounding  a  service  member’s  disabling  medical  condition  under  the 
regulations  of  the  respective  military  department,  an  LOD  determination  is  made  during  the  physical 
disability  evaluation  phase  to  establish  whether  the  member’s  disability  was  incurred  or  aggravated  while 
the  member  was  in  a  duty  status,  as  defined  in  DoD  Instruction  1332.38  (1996). 
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Factors  that  lead  to  a  Medical  Evaluation  Board  for  active  duty  service  members  in¬ 
clude  the  following: 

•  a  condition  that  may  permanently  interfere  with  service  and/or  require  perma¬ 
nent  assignment  limitations 

•  mental  incompetency  in  managing  personal  affairs 

•  member’s  refusal  of  reasonable  medical  treatment. 

Note  the  following  military  department-specific  differences: 

•  In  the  Department  of  the  Army,  a  service  member  undergoing  treatment  for 
medical  conditions  may  receive  a  period  of  convalescent  leave  during  which  he 
or  she  is  still  considered  a  patient  of  the  MTF;  the  service  member  must  return 
periodically  for  evaluation  by  a  physician.  A  service  member  who  can  return  to 
duty  with  restrictions  is  given  a  profile.  Profiles  are  either  temporary,  to  be 
reevaluated  at  a  given  date,  or  permanent,  to  remain  with  a  service  member  for 
the  remainder  of  his  or  her  career.  In  some  of  these  cases,  a  service  member’s 
ability  to  meet  retention  standards  may  be  questionable,  resulting  in  referral  to  a 
Medical  Evaluation  Board. 

•  In  the  Department  of  the  Navy,  a  service  member  may  receive  up  to  30  days  of 

light  duty”  while  undergoing  treatment  for  a  medically  diagnosed  condition.  If 
the  member  continues  to  need  medical  treatment  at  the  end  of  the  30-day  pe¬ 
riod,  he  or  she  may  be  referred  to  a  Limited  Duty  Board  or  Medical  Evaluation 
Board  for  further  evaluation.  The  member  may  receive  up  to  16  months  of 
Temporary  Limited  Duty  (in  up  to  eight-month  increments)  or  spend  up  to  30 
days  in  Medical  Hold  pending  completion  of  a  Medical  Evaluation  Board  referral 
to  the  PEB. 

•  In  the  Air  Force,  the  Medical  Standards  Branch  assigns  and  removes  "Code  C,” 
an  assignment  limitation  code,  to  a  member’s  personnel  records.  The  system 
monitors  a  service  member  assigned  Code  C  and  generally  examines  his  or  her 
medical  condition  every  one  or  two  years,  depending  on  the  condition.  The  Air 
Force  does  not  limit  the  length  of  time  a  member  may  serve  with  a  Code  C. 

From  the  perspective  of  primary  participants  across  the  military  departments,  the 
date  a  narrative  summary  is  dictated  is  generally  accepted  as  the  date  the  service 
member  enters  the  DES.  However,  from  the  perspective  of  the  service  member,  sub¬ 
stantial  time  may  pass  between  the  date  the  referring  physician  decides  the  service 
member’s  medical  condition  calls  into  question  his  or  her  ability  to  meet  medical 
retention  standards  and  the  date  the  physician  actually  dictates  the  narrative  sum¬ 
mary.  During  this  intervening  time,  the  service  member  schedules  and  awaits  the 
appropriate  specialty  consultations,  the  results  of  the  various  medical  tests,  and  the 
synthesis  of  all  of  his  or  her  pertinent  medical  evidence  into  a  narrative  summary.^ 


^Numerous  primary  participants  expressed  concern  that  progre.ssion  through  this  preliminary  phase  is 
not  generally  monitored  and  many  service  members  get  "lost"— that  is,  delayed— in  the  system  while 
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The  narrative  summary  initiates  the  service  member’s  disability  case  file,  which  is 
generally  referred  to  as  the  "medical  board”  or  the  "MEB.” 

The  narrative  summary  documents  the  full  clinical  information  for  all  of  the  service 
member’s  medical  conditions  and  states  whether  any  of  them  is  cause  for  referral 
into  the  DES.  The  summary  includes  a  medical  history,  results  of  appropriate  physi¬ 
cal  examinations,  and  medical  test  results.  It  synthesizes  all  pertinent  medical  evi¬ 
dence  from  all  appropriate  medical  and  surgical  consultations  into  one  comprehen¬ 
sive  document  together  with  diagnosis,  treatment,  and  prognosis.  It  clearly  describes 
the  service  member’s  current  physical  and/ or  mental  condition  in  enough  detail  for 
the  PEB  to  adjudicate  the  case.  Although  all  narrative  summaries  require  specific  de¬ 
tailed  medical  data  for  PEB  adjudication,  the  following  categories  of  cases  require 
information  in  greater  detail  and  with  more  specificity  than  other  categories  require: 

•  Orthopedic 

•  Neurological/Neurosurgical  (in  particular,  backs) 

•  Ophthalmologic 

•  Pulmonary 

•  Cardiological 

•  Psychiatric 

•  Migraine  headache-related 

•  Fibromyalgia 

•  Rheumatology. 

DoD  Instruction  1332.38  (1996)  encourages  physicians  who  prepare  medical  boards 
for  referral  for  physical  disability  evaluation  to  use  the  Department  of  Veterans 
Administration’s  Physician’s  Guide  for  Disability  Evaluation  to  describe  the  nature 
and  degree  of  severity  of  the  member’s  condition. 

Each  medical  board  contains  numerous  additional  documents,  depending  on  the 
particular  case  and  the  military  department’s  administrative  requirements.  All,  how¬ 
ever,  contain  the  following  nonmedical  documents:  (1)  a  letter  from  the  service 
member’s  commander  describing  the  impact  of  the  service  member’s  medical  con¬ 
dition  on  the  member’s  ability  to  perform  his  or  her  normal  military  duties  and  to 
deploy  or  mobilize,  as  applicable;  (2)  a  copy  of  the  LOD  determination,  when  re¬ 
quired;  (3)  pertinent  personnel  records  as  required  by  the  member’s  service  to  estab¬ 
lish  his  or  her  military  history;  and  (4)  an  official  document  identifying  the  next  of 
kin,  court  appointed  guardian,  or  trustee  when  a  service  member  is  determined 


waiting  for  all  the  required  actions  before  the  physician  dictates  the  narrative  summary.  Some  MTFs 
reportedly  manage  this  otherwise  unmonitored  period  of  time  to  their  advantage  by  requiring  physicians 
to  wait  to  dictate  tbe  narrative  summary  until  the  PEBLO  assembles  all  of  the  medical  and  nonmedical 
documents  that  constitute  a  medical  board.  This  practice  contributes  to  decreased  Medical  Evaluation 
Board  processing  time,  which  is  reported  to  the  OSD. 

l^Department  of  Defense  Instruction  1332.39  (1996)  details  the  exact  type  of  medical  information 
required  for  all  cases. 
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incompetent.  These  additional  documents  may  also  include  previous  medical 
boards,  the  member’s  rebuttal,  and  the  Comprehensive  Clinical  Evaluation  Protocol 
evaluation  or  waiver,  if  appropriate. 

Note  the  following  military  department-specific  differences: 

•  The  Departments  of  the  Army  and  Navy  medical  boards  include  Standard  Form 
88/Standard  Form  93,  Report  of  Medical  History. 

•  The  Department  of  the  Navy  medical  boards  may  include  previous  Limited  Duty 
Boards  and  physician  surrebuttals. 


Patient  administrators  at  the  MTFs  generally  assist  PEBLOs  in  compiling  the  neces¬ 
sary  medical  and  nonmedical  documents  that  constitute  the  comprehensive  dis¬ 
ability  case  file— that  is,  the  medical  board— before  forwarding  it  to  the  PEB. 

Note  the  following  military  department-specific  differences: 

•  The  Army  job  titles  for  what  this  report  refers  to  as  "patient  administrators”  in¬ 
clude  "patient  administration  staff  members”  and  "MEB  clerks”  that  assist 
PEBLOs. 

•  At  the  Department  of  the  Navy  medical  centers,  known  as  the  "Big-8,”  PEBLOs 
focus  on  counseling  service  members  whereas  patient  administrators  compile 
the  documentation  for  the  medical  boards. 

Dictating  the  narrative  summary  triggers  initial  counseling  by  the  PEBLOs. 

Note  the  following  military  department-specific  differences: 

•  Most  Army  PEBLOs  are  Army  civilian  employees,  assigned  to  positions  in  the 
continental  United  States  (CONUS).  The  Army  typically  assigns  noncommis¬ 
sioned  officers  with  backgrounds  in  patient  administration  to  its  limited  medical 
PEBLO  positions  outside  CONUS.  Army  PEBLOs  are  appointed  by  and  work  for 
the  MTF  commander,  not  the  Physical  Evaluation  Board. 

•  The  Department  of  the  Navy  assigns  senior  enlisted  members  (E-7  or  above)  with 
backgrounds  in  patient  administration  to  PEBLO  positions  at  the  eight  Navy 
medical  centers.  PEBLOs  at  the  Big-8  are  the  only  PEBLOs  in  any  of  the  military 
departments  that  work  for  the  Physical  Evaluation  Board.  The  Department  of  the 
Navy  also  assigns  equivalent  Navy  civilian  employees  and  enlisted  members— 
called  disability  evaluation  counselors” — from  a  wide  range  of  diverse  special¬ 
ties,  such  as  nuclear  machinist’s  mate,  electrician,  postal  worker,  electronic 
warfare  technician,  gas  turbine  electronic  technician,  and  aviation  ordinance 
technician,  to  collateral  PEBLO  duty  at  smaller  MTFs;  they  work  for  the  MTF 
commander. 

•  The  Air  Force  also  typically  assigns  enlisted  service  members  with  a  background 
in  the  patient  affairs  medical  career  field  to  PEBLO  duty,  although  it  recently  be¬ 
gan  reorganizing  the  PEBLO  function  from  patient  affairs  in  the  MTFs  to  flight 
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medicine,  which  actually  performs  the  medical  examinations.  Some  Air  Force 
PEBLOs  are  also  Air  Force  civilian  employees.  Air  Force  PEBLOs  also  work  for 
MTF  commanders. 

The  PEBLOs  advise  service  members  regarding  what  to  expect  throughout  all  phases 
of  the  disability  evaluation  process,  the  significance  and  consequences  of  the  deter¬ 
minations  that  are  made,  and  the  service  member’s  rights,  benefits,  and  entitle¬ 
ments. 

Note  the  following  military  department-specific  differences: 

•  The  Army  makes  every  effort  to  send  members  for  pre-separation  counseling  90 
days  prior  to  separation. 

•  The  Department  of  the  Navy  PEBLOs  funnel  service  members  through  the  Navy 
Transition  Assistance  Program  (TAP).  PEBLOs  present  a  Disability  Transition 
Assistance  Program,  which  is  Part  1  of  the  TAP.  Part  2  is  mandated  by  law  and 
managed  by  commanders. 

•  In  the  Air  Force,  as  soon  as  it  is  evident  that  a  service  member  will  meet  a 
Medical  Evaluation  Board,  the  PEBLO  refers  the  member  to  the  Military 
Personnel  Flight  for  pre- separation  counseling  to  satisfy  the  requirements  of  10 
U.S.C.  1142,  although  final  disposition  within  the  DES  is  unknown. 

A  Medical  Evaluation  Board— a  clinical  body  of  two  or  three  physicians  at  an  MTF— 
reviews  the  narrative  summary  and  supporting  addenda.  One  Medical  Evaluation 
Board  member  is  a  psychiatrist  when  a  psychiatric  condition  is  under  examination. 

Note  the  following  militcuy  department-specific  differences: 

•  The  Army  Medical  Evaluation  Boards  consist  of  two  or  three  physicians  plus  a 
reviewing  authority.  They  do  not  convene  a  “group  board”;  they  pass  the  medical 
boards  among  the  designated  members,  one  at  a  time. 

•  Department  of  the  Navy  Medical  Evaluation  Boards  also  act  by  passing  a  medical 
board  from  one  physician  to  the  next  until  all  three  designated  members  have 
reviewed  it. 

•  Only  the  Air  Force  actually  convenes  a  group  of  three  staff-rank  physicians  (that 
is,  not  interns  or  residents)  in  one  place  at  one  time  to  act  on  narrative  sum¬ 
maries,  with  the  MTF  commander  or  designee  as  an  approving  authority. 

The  Medical  Evaluation  Board  documents,  under  departmental  regulations,  the  ser¬ 
vice  member’s  medical  status  and  duty  limitations  based  on  the  medical  diagnosis 
and  prognosis  found  in  the  narrative  summary.  The  Medical  Evaluation  Board  eval¬ 
uates  and  reports  on  the  (1)  diagnosis;  (2)  prognosis  for  return  to  full  duty;  (3)  plan 
for  further  treatment,  rehabilitation,  or  convalescence;  (4)  estimated  length  of  time 
the  disabling  condition  will  exist;  and  (5)  medical  recommendations  for  the  disposi¬ 
tion  of  the  service  member.  The  Medical  Evaluation  Board  determines  if  a  reasonable 
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doubt  exists  of  a  service  member’s  ability  to  meet  medical  retention  standards  to 

perform  military  duties. 

Note  the  following  military  department-specific  differences: 

•  Only  the  Army  Medical  Evaluation  Boards  determine  the  service  member’s  abil¬ 
ity  to  meet  medical  retention  standards  only  for  his  or  her  current  military  occu¬ 
pational  specialty. 

•  The  Department  of  the  Navy  Medical  Evaluation  Board  makes  a  clear  statement 
of  its  opinion  that  the  member’s  condition  does  or  does  not  render  the  member 
"unable  to  continue  naval  service  by  reason  of  physical  impairment”  (Secretary 
of  the  Navy  Instruction  1850.4D,  1998,  p.  3-59). 

•  The  Air  Force  Medical  Evaluation  Board  relates  the  member’s  defects,  capabili¬ 
ties,  limitations,  and  prognosis  to  the  military  environment.  Members  must  be 
able  to  perform  military  service  in  such  a  manner  as  to  reasonably  fulfill  the  pur¬ 
pose  of  their  employment  on  active  duty  (U.S.  Department  of  the  Air  Force 
Physical  Disability  Division,  1999,  p.  9). 

The  Medical  Evaluation  Board  recommends  a  case  disposition  based  solely  on  a 

records  review.  The  Medical  Evaluation  Board  may  recommend 

•  return  to  duty 

•  referral  to  the  PEB 

•  the  case  be  returned  to  the  physician(s)  for  further  evaluation,  treatment,  or 
clarification 

•  referral  to  the  parent  service  for  review  and  disposition. 

Note  the  following  military  department-specific  differences: 

•  Department  of  the  Navy  Medical  Evaluation  Boards  may  also  recommend  a  pe¬ 
riod  of  “light  duty”  or  a  period  of  temporary  limited  duty. 

•  If  an  Air  Force  Medical  Evaluation  Board  finds  an  Air  Force  service  member  tem¬ 
porarily  disqualified  for  worldwide  duty,  it  may  forward  the  case  to  the  Medical 
Standards  Branch  for  review  and  approval.  The  Medical  Standards  Branch  may 
direct  further  observation  or  treatment.  In  these  cases,  the  Medical  Standards 
Branch  gives  the  service  member  a  Temporary  4  profile  (4-T)  and  the  case  is  re¬ 
considered  at  a  later  date.  The  service  member  may  remain  on  a  4-T  profile  for  a 
maximum  of  one  year.  A  service  member  who  remains  disqualified  for  worldwide 
duty  at  the  end  of  one  year  on  4-T  must  be  processed  for  Medical  Evaluation 
Board/PEB  evaluation  (U.S.  Department  of  the  Air  Force  Instruction  48-123 
1994). 


‘^Throughout  this  report,  the  term  "records  review"  means  that  the  service  member  does  not  appear 
before  the  decisionmaking  body;  only  the  written  record  of  the  service  member — the  disability  case  file 
referred  to  as  the  ‘'medical  board”— represents  the  service  member’s  case. 
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The  Medical  Evaluation  Board  approving  authority  reviews  all  Medical  Evaluation 
Board  decisions  recorded  in  the  medical  boards  before  forwarding  to  the  PEB.  This 
approving  authority  is  a  senior  physician,  generally  assigned  or  delegated  by  the  MTF 
commander,  and  is  not  considered  a  member  of  the  Medical  Evaluation  Board. 

Note  the  following  military  department-specific  difference: 

•  The  Department  of  the  Navy  title  for  the  Medical  Evaluation  Board  approving 
authority  position  is  “M.E.B.  convening  authority,”  although  the  Department  of 
the  Navy  Medical  Evaluation  Board  does  not  physically  convene  in  one  place  to 
review  medical  boards. 


AA/hen  a  Medical  Evaluation  Board  report  expresses  a  reasonable  doubt  of  a  service 
member’s  ability  to  fulfill  the  duties  of  his  or  her  office,  grade,  rank,  or  rating,  the 
MTF  refers  the  medical  hoard  to  the  PEB  for  a  determination  of  fitness  and  a  disabil¬ 
ity  rating  for  those  found  unfit. 

DoD  Instruction  1332.38  (1996)  establishes  the  following  time  requirement  goal  for 
the  medical  evaluation  phase  of  the  DES:  “When  a  physician  initiates  a  Medical 
Evaluation  Board,  the  processing  time  should  normally  not  exceed  30  days  from  the 
date  the  Medical  Evaluation  Board  report  is  dictated  to  the  date  it  is  received  hy  the 
Physical  Evaluation  Board.” 

Note  the  following  military  department-specific  difference: 

•  Secretary  of  the  Navy  Instruction  1850.4D  (p.  1-11)  restates  the  timeliness  goal: 
“Medical  Board  reports  referring  members  to  the  Physical  Evaluation  Board  will 
be  processed,  dictated,  and  received  by  the  Physical  Evaluation  Board  within  30 
days  of  the  attending  physician’s  desire  to  convene  a  medical  board  based  on  the 
doctor’s  opinion  that  the  service  member’s  return  to  full  duty  is  unlikely  and  op¬ 
timal  medical  benefits  have  been  attained.  Delays  of  acceptance  by  the  PEB  for 
completion  of  case  documentation  requirements  are  not  included  within  this  time 
standard”  [emphasis  added]. 

DoD  Instruction  1332.38  establishes  the  following  time  requirement  goal  for  Reserve 
component  service  members  referred  solely  for  a  fitness  determination  on  a  non¬ 
duty-related  condition:  “For  cases  of  Reserve  component  members  referred  solely  for 
a  fitness  determination  on  a  non-duty-related  condition,  processing  time  for  con¬ 
duct  of  Medical  Evaluation  Board  or  physical  examination  shall  not  exceed  90  calen¬ 
dar  days.” 

Phase  2.  Physical  Disability  Evaluation  and  Disposition 

The  PEB  conducts  the  Physical  Disability  Evaluation  process;  the  process  consists  of 
two  levels  of  adjudication:  Informal  PEB  adjudication  and  Formal  PEB  adjudication. 
The  Informal  PEB  conducts  a  records  review  and  issues  findings  and  recommenda¬ 
tions.  Service  members  found  unfit  who  choose  to  appeal  the  findings  and  recom¬ 
mendations  of  the  Informal  PEB  have  an  opportunity  to  present  their  case  in  person 
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with  legal  representation  at  the  Formal  PEB,  which  then  issues  findings  and 
recommendations. 

PEB  administrative  action  officers  are  the  focal  point  for  quality  assurance  during 
this  phase.  They  (1)  receive  medical  boards  from  MTFs,  log  them  in,  quality-check 
them  for  administrative  sufficiency,  send  insufficient  medical  boards  back  to  the  re¬ 
ferring  MTF,  and  route  sufficient  ones  to  the  Informal  PEB;  (2)  notify  appropriate 
service  headquarters  of  pending  PEB  actions  on  service  members;  and  (3)  forward 
medical  boards  of  appealed  cases  to  the  Formal  PEB.  The  following  sections  of  this 
chapter  outline  the  operations  of  the  two  levels  of  PEB  adjudication— Informal  PEB 
adjudication  and  Formal  PEB  adjudication,  plus  final  disposition  from  both. 

Note  the  following  military  department-specific  differences: 

•  The  Army  job  titles  for  what  this  report  refers  to  as  “PEB  administrative  action 
officer  include  case  analyst”  and  “recorder.”  Recorders  are  noncommissioned 
officers,  warrant  officers,  or  civilians  of  equivalent  grades  who  work  for  the  PEB. 

•  “Recorder”  is  also  the  job  title  in  the  Department  of  the  Navy. 

•  The  Air  Force  job  title  is  “action  officer”;  nine  GS-07s  and  one  technical  sergeant 
serve  as  action  officers  to  support  the  Informal  PEB.  Unlike  the  other  military 
departments,  each  action  officer  manages  an  individual  case  from  the  time  it  is 

the  Informal  PEB  until  the  case  is  closed,  a  period  which  may  cover 
several  years,  depending  on  the  stability  of  the  member’s  condition  and  the  level 
of  appellate  review  sought. 


Informal  Physical  Evaluation  Board.  The  Informal  PEB  consists  of  three  voting 
members,  including  at  least  one  physician,  and  one  nonmedical  officer.  The  physi- 
cian(s)  interpret(s)  the  diagnosis  and  prognosis  from  the  Medical  Evaluation  Board. 
The  nonmedical  officer— typically  a  personnel  officer— interprets  the  impact  on  the 
service  member  s  unit  from  the  member’s  inability  to  perform  his  or  her  duties  as  a 
result  of  the  condition  or  impairment.  A  Reserve  component  officer  fills  one  of  the 
three  voting  positions  when  adjudicating  a  Reserve  component  case.  When  the  board 
members  cannot  agree  on  findings  or  recommendations,  the  dissenting  member 
may  write  a  minority  opinion  that  becomes  part  of  the  medical  board. 

Note  the  following  military  department-specific  differences: 

•  Army  Informal  PEB  composition  normally  includes  a  nonmedical  officer  presi¬ 
dent  (0-6),  one  personnel  management  officer,  and  one  physician  who  may  be 
either  civilian  or  military.  The  president  and  personnel  management  officer  may 
be  of  any  branch  except  the  special  branches.  The  personnel  management  officer 
is  usually  a  Reserve  or  National  Guard  member.  A  Reserve  component  officer, 
otherwise  qualified  for  PEB  duty,  serves  on  the  informal  board  when  it  evaluates 
Reserve  component  cases.  Likewise,  female,  minority,  or  enlisted  representation 
on  the  Formal  PEB  is  provided,  when  possible,  upon  request.  The  same  members 
constitute  both  the  Informal  and  Formal  PEBs,  which  means  the  same  members 
may  adjudicate  the  same  case  on  two  different  levels. 
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•  Department  of  the  Navy  Informal  PEB  membership  consists  of  one  medical  offi¬ 
cer  and  two  line  officers,  usually  a  Navy  and  Marine  Corps  officer.  All  members 
are  senior  military  officers,  0-6  preferred.  One  of  the  two  line  officers  acts  as  the 
Informal  PEB  administrator,  preferably  the  line  officer  from  the  member’s 
service  (Navy  or  Marine  Corps). 

•  Air  Force  Informal  PEB  membership  consists  of  two  medical  officers  (0-6s)  and 
one  line  officer,  generally  a  personnel  officer  (0-5  or  0-6),  who  is  designated 
Informal  PEB  president. 

The  Informal  PEB  determines  whether  the  service  member  is  eligible  for  full  adjudi¬ 
cation  or  only  a  fitness  finding  for  Reserve  component  non-duty  related  cases.  It 
evaluates  each  case  and  issues  a  finding  of  each  service  member’s  fitness  to  perform 
the  duties  of  his  or  her  office,  grade,  rank,  or  rating  on  the  basis  of  the  preponderance 
of  the  evidence  in  the  medical  board. 

Note  the  following  military  department-specific  differences: 

•  The  Army  Informal  PEB  issues  a  finding  of  the  service  member’s  fitness  to  per¬ 
form  the  duties  of  his  or  her  "office,  grade,  rank,  or  rating  and  military  occupa¬ 
tional  specialty.’’ 

•  Air  Force  Informal  PEB  issues  a  finding  of  the  service  member’s  fitness  to  per¬ 
form  the  duties  of  his  or  her  “office,  grade,  or  rank.” 


In  each  case,  the  Informal  PEB  weighs  the  nature  and  degree  of  the  service  member’s 
condition  or  impairment  as  presented  in  the  medical  board  against  the  requirements 
and  duties  expected  of  the  service  member’s  office,  grade,  rank,  or  rating,  and  the 
commander’s  assessment  of  the  service  member’s  duty  performance. 

The  Informal  PEB  considers  the  following  compensability  criteria: 

•  Any  injury  or  disease  discovered  after  a  service  member  enters  active  duty,  with 
the  exception  of  congenital  and  hereditary  conditions,  is  presumed  to  have  been 
incurred  in  the  line  of  duty. 

•  Presumption  that  service  incurred  or  service  aggravated  condition,  and 
overcoming  presumption. 

•  Line  of  duty  determination  (depending  on  the  case — administrative,  informal,  or 
formal). 

Note  the  following  military  department-specific  difference  (to  the  LOD 
Determination): 

-  The  Naval  Reserve  uses  the  term  Notice  of  Eligibility  for  the  LOD 
Determination. 

•  Standard  of  proximate  result  applies  to  Reserve  component  members  whose  dis¬ 
ability  originated  prior  to  September  24, 1996. 
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•  Presumption  of  fitness,  and  overcoming  presumption.  Members  with  retirement 
dates  and  members  who  face  higher  tenure  restrictions  are  presumed  fit. 
Because  these  members  qualify  for  length-of-service  retirement,  the  potentially 
disabling  condition  is  presumed  not  to  be  a  reason  for  "early  firing.” 

•  Noncompliance  (refusal  of  treatment). 

Based  on  the  information  in  the  service  member’s  medical  board,  the  Informal  PEB 

may  find  the  member  fit  or  unfit.  The  military  departments  each  rely  on  different  fit¬ 
ness  criteria. 

Note  the  following  military  department-specific  differences: 

•  The  Army  PEB  (both  Informal  and  Formal)  determines  fitness  based  upon 
whether  the  record  of  evidence  shows  that  the  medical  condition  does  or  does 
not  preclude  reasonable  performance  of  the  duties  required  of  the  service  mem¬ 
ber’s  office,  grade,  rank,  or  rating.  It  relies  heavily  on  the  performance  data  pro¬ 
vided  by  the  service  member’s  immediate  commander  (DoD  Instruction  1332.38, 
1996,  Part  3,  paras.  B  and  C  [published  version),  and  DoD  Instruction  1332.38, 
1996,  Part  3,  paras.  E3.P3.2  and  E3.P3.3  [electronic  versionl:’^  aR  635-40,  1990, 
para.  4-19d[2]). 

•  The  Department  of  the  N avy  determines  fitness  by  relating  the  nature  and  degree 
of  physical  disability  of  the  member  to  the  requirements  and  duties  that  member 
may  reasonably  be  expected  to  perform  in  his  or  her  office,  grade,  rank,  or  rating. 
It  published  its  fitness  standards  and  criteria  in  Secretary  of  the  Navy  Instruction 
1850.4D,  1998,  pp.  3-13  through  3-17. 

•  The  Air  Force  relies  on  the  standards  and  criteria  for  determining  fitness  in  DoD 
Directive  1332.18,  para.  C.3  (para.  3.3  in  the  electronic  version)  (U.S.  Department 
of  the  Air  Force  Instruction  36-3212,  1998,  p.  17).  The  Air  Force  Informal  PEB  may 
express  its  opinion  concerning  possible  reclassification,  but  does  not  have  the 
authority  to  direct  reclassification,  establish  physical  profile  limitations  or  direct 
assignments  (U.S.  Department  of  the  Air  Force  Physical  Disability  Division,  1999 
p.  12). 


If  the  Informal  PEB  finds  a  service  member  unfit  and  the  service  member  does  not 
have  an  LOD  Determination  of  No  (LOD-No)  or  a  condition  that  existed  prior  to 
service,  the  Informal  PEB  assigns  a  code  and  rates  the  service  member’s  degree  of 
disability  using  the  VASRD,  the  DoD  Instruction  1332.39,  Enclosure  3,  or  the  current 
analogous  codes  established  by  a  group  of  physicians  from  all  three  military  depart- 


1 

The  Department  of  Defense  Directive  (DoDD)  1332.18  and  Department  of  Defense  Instruction  (DoDI) 
1332.38  documents  posted  on  the  DoD  Web  site  use  a  different  paragraph  numbering  system  than  the 
published  paper  documents.  Primary  participants  reported  that  service  member  customers  generally  have 
easier  access  to  the  electronic  version  on  the  Web  than  to  the  published  version,  whereas  primary 
participants  who  work  with  the  system  on  a  daily  basis  rely  almost  exclusively  on  the  published  paper 
versions.  As  a  result,  primary  participants  who  respond  to  customer  inquiries  based  on  the  electronic 
version  of  the  DoD  Directive  and  the  DoD  Instruction  must  translate  the  paragraph  numbers  in  their 
published  paper  version  to  the  paragraph  numbers  in  the  customer’s  electronic  version. 
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ments  and  distributed  to  augment  those  published  in  Enclosure  3.  The  analogous 
codes  supplement  VASRD  codes,  which  do  not  include  all  possible  impairments  that 
result  from  combat  or  many  current  medical  diagnoses. 

The  range  of  recommendations  available  to  an  Informal  PEB  for  a  service  member 
found  unfit  to  perform  the  duties  of  the  member’s  office,  grade,  rank,  or  rating  is  as 
follows: 

•  Stable  condition,  unfit 

-  Discharge  with  severance  pay  if  the  disability  is  rated  less  than  30  percent  and 
member  has  less  than  20  years  of  service 

-  Discharge  without  severance  pay  (in  cases  of  LOD-No  or  Existed  Prior  to  Service 
[EPTS]) 

-  Retire  for  disability  if  the  disability  is  rated  30  percent  or  more  or  member  has 
more  than  20  years  of  service  and  is  eligible  for  retirement 

•  Unstable  condition,  unfit 

-  Place  on  Temporary  Disability  Retired  List  if  the  disability  is  rated  30  percent  or 
more  or  the  member  has  more  than  20  years  of  service  and  is  eligible  for 
retirement. 

The  range  of  recommendations  available  to  the  Informal  PEB  for  a  service  member 
found  fit  to  perform  the  duties  of  the  member’s  office,  grade,  rank,  or  rating,  if  the 
condition  is  stable,  is  as  follows: 

•  Fit 

-  Return  to  duty 

-  Remove  from  TDRL  and  return  to  duty. 

All  three  military  departments  exercise  an  administrative  process  for  granting  light  or 
limited  duty  to  a  service  member  who  is  found  fit  but  requires  additional  time  to 
heal. 

When  the  Informal  PEB  finds  a  service  member  fit,  the  PEB  administrative  action  of¬ 
ficers  route  the  medical  board  back  through  the  appropriate  administrative  channels 
to  the  MTF  and  notify  the  service  member's  PEBLO.  The  PEBLO  notifies  the  service 
member  of  the  findings,  recommended  disposition,  and  appeal  options. 

Note  the  following  military  department-specific  differences: 

•  An  Army  service  member  has  ten  calendar  days  to  make  a  decision  regarding  the 
Informal  PEB  findings  and  recommendations.  A  member  found  fit  may  elect 
either  of  the  following  options: 

-  Concur 

-  Nonconcur,  with  or  without  rebuttal. 
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If  the  service  member  concurs,  the  PEB  president  approves  the  proceedings  for 
the  Secretary  of  the  Army.  The  PEB  recorder  forwards  the  medical  board  to  the 
Physical  Disability  Branch  within  the  Army  Physical  Disability  Agency  for  final 
disposition. 

Unless  a  minority  report  was  entered,  if  the  service  member  nonconcurs  with  the 
finding  without  submitting  a  rebuttal,  the  PEB  president  has  approval  authority 
for  the  Secretary  of  the  Army  and  forwards  the  case  to  the  Physical  Disability 
Branch  for  final  disposition.  The  Army  Physical  Disability  Agency  must  approve 
all  cases  that  include  a  minority  report  before  final  disposition.  If  the  service 
member  nonconcurs  and  submits  a  statement  or  rebuttal  to  the  recommended 
findings  without  asking  for  a  formal  hearing,  the  PEB  president  responds  in 
writing  to  the  service  member,  normally  within  three  days.  If  the  service  mem¬ 
ber’s  rebuttal  does  not  result  in  a  change  to  the  Informal  PEB  findings,  the  re¬ 
sponse  explains  the  Informal  PEB’s  decision  to  adhere  to  the  earlier  findings.  The 
service  member  is  advised  that  the  rebuttal  will  be  included  in  the  medical  board 
and  considered  in  the  review  action  by  the  Army  Physical  Disability  Agency.  A 
copy  of  the  PEB  president’s  letter  is  included  in  the  medical  board  that  is  for¬ 
warded  to  the  Army  Physical  Disability  Agency  for  final  review. 

•  A  Department  of  the  Navy  service  member  has  15  calendar  days  in  which  to 
make  a  decision  regarding  the  Informal  PEB  findings  and  recommendations;  ac¬ 
ceptance  is  presumed  on  the  sixteenth  day  after  the  receipt  of  findings.  A  mem¬ 
ber  found  fit  may  elect  either  of  the  following  options: 

-  Accept  the  fit  finding  and  continue  service 

-  Disagree  with  the  finding  and  request  reconsideration  by  the  Informal  PEB. 

Reconsideration  may  relate  to  the  same  diagnosis  or  a  new  diagnosis.  A  member 
offering  new  medical  information,  or  a  significant  nonmedical  assessment  that 
was  not  previously  available  or  considered,  is  eligible  to  have  the  Informal  PEB 
reconsider  the  case.  The  member  must  also  present  a  new  nonmedical  assess¬ 
ment.  In  requesting  reconsideration,  the  member  must  also  submit  a  statement 
regarding  his  or  her  desire  for  a  Formal  PEB  if  the  findings  are  unchanged.  If  the 
new  information  does  not  change  the  results  of  the  Informal  PEB  finding,  the 
PEB  president  may  grant  a  member  a  Formal  PEB.  The  member  found  fit  does 
not  have  a  right  to  a  Formal  PEB.  If  the  member  does  not  request  a  hearing,  or  if 
the  hearing  request  is  denied,  the  Informal  PEB  findings  become  final.  If,  upon 
reconsideration,  the  finding  is  changed  to  unfit,  the  member  receives  new  notifi¬ 
cation  and  is  presented  with  the  applicable  options. 

•  Air  Force  members  found  fit  do  not  have  a  right  to  an  appeal  process  because 
they  have  not  been  “fired.”  However,  the  Informal  PEB  will  review  the  cases 
again  at  the  request  of  the  commander  of  the  referring  MTF,  if  the  commander 
believes  that  important  evidence  was  omitted  from  the  previous  medical  board 
that  was  sent  to  the  Informal  PEB. 
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When  the  Informal  PEB  finds  the  service  member  unfit,  it  determines  ivhether  the 
service  member  is  eligible  for  other  special  considerations,  such  as  permanent  lim¬ 
ited  duty,  as  an  exception  to  policy. 

Note  the  foUowing  military  department-specific  difference: 

•  The  Army  Informal  PEB  determines  whether  the  service  member  is  eligible  for 
Temporary  Early  Retirement  Authority  or  Reserve  component  member  early 
qualification  for  retired  pay  at  age  60. 

•  Only  the  Department  of  the  Navy  limits  the  amount  of  limited  duty  time  awarded 
by  the  PEB,  which  together  with  the  service  headquarters  has  authority  to  grant 
permanent  limited  duty  of  60  days  or  less  for  Marines  and  90  days  or  less  for 
Navy  members. 

The  Informal  PEB  determines  if  the  disabling  condition  meets  the  criteria  for 
Instrumentality  of  War  issues,  such  as  exemption  of  disability  retired  or  severance 
pay  from  gross  federal  income  tax,  eligibility  for  civil  service  preference  status,  and 
exemption  from  the  Dual  Compensation  Act.^^ 

When  the  Informal  PEB  finds  a  service  member  unfit,  the  PEB  administrative  action 
officers  notify  the  PEBLO  who  counsels  the  service  member,  in  person  when  possi¬ 
ble,  on  the  findings,  disposition  recommendation,  implications,  and  appeal  options. 
Depending  on  the  option  the  service  member  elects,  the  PEB  administrative  action 
officers  route  the  medical  board  to  the  appropriate  review  authority,  personnel 
headquarters,  or  the  Formal  PEB.  Depending  on  military  department  policy,  the  ser¬ 
vice  member  has  from  three  duty  days  to  15  calendar  days  to  elect  options. 

Note  the  following  military  department-specific  differences: 

•  An  Army  service  member  has  ten  calendar  days  to  make  a  decision  regarding  the 
Informal  PEB  findings  and  recommendations.  A  member  found  unfit  may  elect 
from  among  the  following  options: 

-  Concur 

-  N onconcur  with  or  without  rebuttal 

-  Demand  Formal  PEB  (unfit  findings  only). 

An  Army  Formal  PEB  is  a  new  hearing;  it  does  not  start  with  or  refine  the  findings 
of  the  Informal  PEB. 

If  the  service  member  concurs,  the  PEB  president  approves  the  proceedings  for 
the  Secretary  of  the  Army.  The  PEB  recorder  forwards  the  medical  board  to  the 
Physical  Disability  Branch  within  the  Army  Physical  Disability  Agency  for  final 
disposition. 


^^The  Dual  Compensation  Act  prohibited  military  officer  retirees  from  collecting  full  military  retirement 
pay  in  addition  to  full  pay  as  a  federal  civilian  employee,  so-called  double-dipping.  During  the  course  of 
this  study,  the  1999  National  Defense  Authorization  Act  for  fiscal  year  2000  repealed  the  reduction  in 
retired  pay  for  military  retirees  employed  in  civilian  positions,  effective  October  1, 1999. 
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Unless  a  minority  report  was  entered,  if  the  service  member  nonconcurs  with  the 
finding  without  submitting  a  rebuttal,  the  PEB  president  has  approval  authority 
for  the  Secretary  of  the  Army  and  forwards  the  case  to  the  Physical  Disability 
Branch  for  final  disposition.  The  Army  Physical  Disability  Agency  must  approve 
all  cases  that  include  a  minority  report  before  final  disposition.  If  the  service 
member  nonconcurs  and  submits  a  statement  or  rebuttal  to  the  recommended 
findings  without  asking  for  a  formal  hearing,  the  PEB  president  responds  in 
writing  to  the  service  member,  normally  within  three  days.  If  the  service  mem¬ 
ber’s  rebuttal  does  not  result  in  a  change  to  the  Informal  PEB  findings,  the  re¬ 
sponse  explains  the  Informal  PEB’s  decision  to  adhere  to  the  earlier  findings.  The 
service  member  is  advised  that  the  rebuttal  will  be  included  in  the  medical  board 
and  considered  in  the  review  action  by  the  Army  Physical  Disability  Agency.  A 
copy  of  the  PEB  president’s  letter  is  included  in  the  medical  board  that  is  for¬ 
warded  to  the  Army  Physical  Disability  Agency  for  final  review.  If  the  service 
member  nonconcurs  with  the  findings  and  recommendations  with  a  statement 
of  rebuttal  and  demands  a  formal  hearing,  the  PEB  may  reconsider  its  findings 
and  recommendations  in  light  of  the  service  member’s  statement  of  rebuttal.  If 
the  Informal  PEB  agrees  with  the  service  member  and  modifies  the  findings  and 
recommendations,  the  PEB  sends  the  amended  findings  to  the  member’s  PEBLO. 
The  PEBLO  then  notifies  the  service  member  of  the  change.  The  service  member 
has  ten  calendar  days  to  make  his  or  her  new  election.  If  the  service  member  ac¬ 
cepts  the  revised  findings,  the  case  is  forwarded  to  the  Physical  Disability  Branch 
for  final  disposition.  If  the  service  member  does  not  accept  the  revised  findings 
or  the  Informal  PEB  does  not  change  its  earlier  findings,  the  case  is  scheduled  for 
a  formal  hearing. 

•  A  Department  of  the  Navy  service  member  has  15  calendar  days  to  make  a 
decision  regarding  the  Informal  PEB  findings  and  recommendations;  acceptance 
is  presumed  on  the  sixteenth  day  after  the  receipt  of  findings.  A  member  found 
unfit  may  do  any  of  the  following: 

Unconditionally  accept  the  findings 

-  Conditionally  accept  the  findings 

-  Demand  a  hearing  before  the  Formal  PEB. 

In  the  case  of  unconditional  acceptance,  the  case  is  forwarded  to  the  PEB  presi¬ 
dent  who  issues  a  Notice  of  Decision  to  the  appropriate  service  headquarters.  In 
the  case  of  conditional  acceptance,  the  member  agrees  to  accept  the  findings  if 
the  condition  requested  is  met  (such  as  a  specified  period  of  permanent  limited 
duty  or  a  specified  separation  or  retirement  date).  When  filing  a  conditional  ac¬ 
ceptance,  the  member  must  indicate  if  he  or  she  desires  a  Formal  PEB  if  the 
condition  is  not  met.  A  member  found  unfit  who  nonconcurs  with  the  Informal 
PEB  findings  may  demand  a  Formal  PEB  hearing. 

•  An  Air  Force  service  member  has  three  duty  days  to  decide  whether  to  accept  or 
appeal  the  recommendations.  A  member  found  unfit  may  do  either  one  of  the 
following: 

-  Agree  with  the  findings 
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-  Disagree  with  the  findings  and  request  a  Formal  PEB  hearing  (U.S.  Department 
of  the  Air  Force,  Physical  Disability  Division,  p.  1 1). 

If  the  member  accepts  the  finding  and  recommendations,  he  or  she  signs  a  form 
that  is  sent  back  to  the  Disability  Operations  Branch  and  the  medical  board  is 
forwarded  to  the  Secretary  of  the  Air  Force  Personnel  Council.  Final  disposition 
includes  outprocessing  the  service  member  from  the  Air  Force.  If  the  member 
decides  to  appeal  the  recommendations,  the  action  officer  assigned  to  the  case 
schedules  an  appointment  for  the  Formal  PEB  within  two  to  three  weeks,  for¬ 
wards  the  medical  board  to  the  Formal  PEB,  and  advises  the  PEBLO  at  the  refer¬ 
ring  MTF. 

In  two  of  the  three  military  departments’  DESs,  the  officer  (0-6)  in  charge  of  the  de¬ 
partment’s  PEB  board  process  (called  the  “PEB  board  approving  authority”  in  this  re¬ 
port)  reviews  the  Informal  PEB  findings  and  disposition  recommendations  for  every 

medical  board. 

Note  the  following  military  department-specific  differences: 

•  No  one  person  reviews  all  Army  Informal  PEB  findings  and  recommendations. 
However,  the  Army  Physical  Disability  Agency  is  responsible  for  reviewing  and 
confirming  Informal  PEB  actions.  The  Army  Physical  Disability  Agency  reviews 
those  cases  in  which  the  service  member  disagrees  with  the  findings  of  the 
Informal  PEB  and  submits  a  rebuttal.  If  the  agency  changes  the  findings  of  the 
Informal  PEB  and  the  service  member  nonconcurs  with  a  rebuttal,  the  case  is 
forwarded  to  the  Army  Physical  Disability  Appeal  Board  for  final  decision.  The 
Army  Physical  Disability  Agency  headquarters  also  conducts  mandatory  records 
reviews  for  quality  assurance  of  the  following  cases: 

-  All  general  officers  and  medical  corps  officers  found  unfit 

-  All  cases  in  which  the  service  member  nonconcurred,  with  or  without  a  rebuttal, 
and  consideration  of  the  rebuttal  did  not  result  in  a  change  in  PEB  findings  and 
recommendations 

-  All  cases  in  which  a  PEB  member  submitted  a  minority  report 

-  All  cases  of  members  assigned  to  the  Army  Physical  Disability  Agency 

-  Any  case  previously  reviewed 

-  Command  directed  quality  reviews  on  special-interest  cases,  such  as  HIV. 

•  The  Army  Physical  Disability  Agency  may  do  the  following: 

-  Concur  with  the  findings  and  recommendations  of  the  Informal  PEB  or  make 
minor  changes  or  corrections  that  do  not  affect  the  recommended  disposition  of 
the  soldier,  or  lower  the  combined  percentage  rating 

-  Return  the  case  to  the  PEB  for  reconsideration,  clarification,  further  investiga¬ 
tion,  a  formal  hearing,  or  other  action  when  the  case  records  show  that  such  ac¬ 
tion  is  in  the  best  interest  of  the  service  member  or  the  Army 
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-  Issue  revised  findings  providing  for  a  change  in  disposition  of  the  service  mem¬ 
ber  or  change  in  the  service  member’s  disability  rating 

-  Refer  the  case  to  the  Army  Physical  Disability  Appeal  Board. 

•  The  Department  of  the  Navy  PEB  president,  who  oversees  all  Navy  PEBs  (both 
Informal  and  Formal),  reviews  all  Informal  PEB  findings  and  disposition  recom¬ 
mendations  and  ensures  each  case  is  administratively  and  legally  sufficient.  If  he 
or  she  concurs,  the  recorder  sends  a  findings  letter  to  the  PEBLO  to  brief  the  ser¬ 
vice  member.  If  the  service  member  does  not  concur,  he  or  she  may  modify  or 
cancel  the  findings  letter  and  notification  of  decision  letters  and  direct  appropri¬ 
ate  substitute  disposition. 

•  The  Chief,  Air  Force  Physical  Disability  Division,  who  oversees  both  the  Informal 
and  Formal  PEBs,  reviews  all  Informal  PEB  case  findings  and  recommendations. 
If  the  Chief,  Air  Force  Physical  Disability  Division  approves  the  PEB  findings  and 
recommendations,  a  PEB  administrative  action  officer  sends  a  findings  letter  to 
the  PEBLO  who  then  informs  the  service  member.  If  the  Chief,  Air  Force  Physical 
Disability  Division  does  not  approve  the  PEB  findings  and  recommendations,  he 
or  she  forwards  the  case  directly  to  the  Formal  PEB. 

A  service  member  found  unfit  who  disagrees  with  the  findings  and  recommendations 
of  the  Informal  PEB  has  a  legal  right,  with  the  assistance  of  an  attorney  at  no  cost  to 
the  member,  to  appeal  his  or  her  case  to  the  Formal  PEB. 

Formal  Physical  Evaluation  Board.  The  Formal  PEB  consists  of  three  voting  mem¬ 
bers,  including  at  least  one  physician,  and  one  nonmedical  officer.  The  physician(s) 
interpret(s)  the  medical  diagnosis  and  prognosis.  The  nonmedical  officer— typically  a 
personnel  officer— interprets  the  impact  of  the  member’s  inability  to  perform  his  or 
her  duties  as  a  result  of  the  condition  or  impairment  on  the  service  member’s  unit.  A 
Reserve  component  officer  fills  one  of  the  three  voting  positions  when  adjudicating  a 
Reserve  component  case. 

A  service  member  may  choose  representation  by  an  attorney  from  the  Office  of  the 
Judge  Advocate  General  at  no  cost  to  the  member.  A  service  member  may  also 
choose  to  hire  a  civilian  attorney  at  his  or  her  own  expense. 

A  service  member  spends  one  to  three  working  days  with  an  attorney  to  prepare  for 
his  or  her  formal  hearing,  depending  on  military  department  policy. 

Note  the  following  military  department-specific  differences: 

•  An  Army  service  member  is  given  a  minimum  of  three  working  days  to  prepare 
his  or  her  case  with  an  attorney.  If  more  time  is  required,  the  service  member  can 
request  an  extension  from  the  PEB  president. 

•  The  Department  of  the  Navy  encourages  a  service  member  to  contact  his  or  her 
attorney  by  phone  in  order  to  start  preparing  the  case  as  soon  as  the  service 
member  decides  to  appeal  and  before  arriving  at  the  Formal  PEB.  The 
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Department  of  the  Navy  service  member  meets  with  his  or  her  attorney  the  day 
before  the  hearing. 

•  An  Air  Force  service  member  is  given  two  days  to  prepare  a  case  with  his  or  her 
attorney  at  the  Formal  PEB  location. 

The  service  member’s  attorney  confers  with  and  fully  advises  the  member  of  legal 
and  other  substantive  considerations  for  his  or  her  case.  The  attorney  represents  the 
service  member  before  the  Formal  PEB,  presenting  information  and  arguments  in 
support  of  the  service  member’s  case.  The  attorney  also  arranges  for  the  presence  of 
desired  witnesses  and  evidence  in  support  of  the  member’s  case,  interviews  wit¬ 
nesses  prior  to  the  formal  hearing,  and  questions  them  during  the  hearing. 

The  Formal  PEB  is  formally  structured  and  nonadversarial  in  nature.  The  proceed¬ 
ings  are  generally  audiotaped. 

Note  the  following  military  department-specific  difference: 

•  The  Air  Force  both  audiotapes  and  videotapes  the  proceedings  and  gives  the 
member  a  copy  of  the  audiotape  before  he  or  she  departs  the  site  of  the  Formal 
PEB. 

The  board  members  review  the  evidence  in  the  medical  board  prior  to  the  formal 
hearing. 

Note  the  following  military  department-specific  difference: 

•  Some  Army  formal  hearings  are  held  via  videoconference. 

The  service  member  is  called  into  the  formal  hearing  chamber  and  sworn  in.  Any 
additional  documents  provided  by  the  service  member  are  entered  into  evidence. 

The  service  member’s  attorney  enters  the  member’s  plea  and  the  board  members 
question  the  service  member  about  his  or  her  medical  condition  of  referral  and  its 
impact  on  the  service  member’s  current  activities  including  work,  school,  and 
recreation. 

Note  the  following  military  department-specific  difference: 

•  Department  of  the  Navy  physician  board  members  may  conduct  medical  exami¬ 
nations  on  the  service  member  during  the  formal  hearing;  Army  boards  do  not 
include  this  practice. 

At  the  end  of  the  questioning,  the  board  provides  the  member  with  an  opportunity  to 
add  any  additional  information  that  would  impact  his  or  her  case.  Upon  completing 
the  open  hearing,  the  board  closes  for  deliberation  and  the  service  member  and  his 
or  her  attorney  leave  the  room.  The  Formal  PEB  members  consult  in  private  to  agree 
upon  a  fitness  decision  and  disposition  recommendation  and  determine  if  any 
additional  information  entered  into  evidence  impacts  Informal  PEB  administrative 
decisions. 
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The  range  of  dispositions  available  to  the  Formal  PEB  is  the  same  as  that  available  to 
the  Informal  PEB, 

Any  dissenting  member  of  a  Formal  PEB  may  submit  a  minority  opinion  citing  par¬ 
ticular  areas  in  which  he  or  she  disagrees  with  the  action  of  the  formal  board.  The 
minority  opinion  becomes  part  of  the  medical  board  case  file. 

Upon  completion  of  the  deliberations,  the  board  reopens,  calls  the  service  member 
and  attorney  back  into  the  hearing  chamber,  and  informs  the  service  member  of  the 
findings  and  recommendations.  The  service  member  and  attorney  then  depart  the 
chamber. 

The  attorney  counsels  the  service  member  regarding  Formal  PEB  findings  and  op¬ 
tions  available  to  the  member  and  recommends  courses  of  action  that  are  most  fa¬ 
vorable  to  the  member  and  that  are  consistent  with  the  letter  and  intent  of  statutes, 
instructions,  and  other  policy  documents  addressing  disability  evaluation  and  ad¬ 
ministration.  The  attorney  advises  the  service  member  and  assists,  if  asked,  in  the 
preparation  and  submission  of  a  request  for  permanent  limited  duty,  and  prepares  or 
assists  in  the  preparation  of  a  rebuttal  at  the  request  of  the  service  member.  In  the 
case  of  incompetent  service  members,  the  attorney  fully  informs  the  court-appointed 
guardian,  or  if  no  guardian  has  been  appointed  by  a  court,  the  service  member’s 
spouse  or  next  of  kin,  as  appropriate,  if  the  wishes  of  the  spouse  or  next  of  kin  do  not 
conflict  with  the  proper  exercise  of  the  responsibilities  of  the  attorney  concerning  the 
member’s  best  interests. 

The  service  member  may  concur  or  nonconcur,  with  or  without  a  rebuttal.  The 
member  has  a  right  to  appeal  the  findings  and  recommendations  of  the  Formal  PEB. 

Depending  on  military  department  policy,  the  service  member  has  zero  to  15  days  to 
elect  options. 

Note  the  following  military  department-specific  differences: 

An  Army  service  member  may  concur  with  the  findings  and  recommendations  or 
nonconcur  with  them,  with  or  without  rebuttal.  He  or  she  has  ten  calendar  days 
to  submit  a  rebuttal.  A  rebuttal  must  be  based  on  one  of  tbe  following  issues: 

-  The  decision  of  the  PEB  was  based  on  fraud,  collusion,  misrepresentation,  or 
other  misconduct 

-  Mistake  of  law 

-  The  service  member  did  not  receive  a  full  and  fair  hearing 

-  Substantial  new  evidence  exists. 

•  When  practical,  a  Department  of  the  Navy  service  member  is  notified  of  the 
findings  either  in  open  session  or  by  his  or  her  attorney,  in  person,  prior  to  leav¬ 
ing  the  Formal  PEB  site.  The  attorney  then  counsels  the  service  member  regard¬ 
ing  the  Formal  PEB’s  recommendations.  A  service  member  is  notified  that  the 
formal  board’s  findings  are  subject  to  review  for  administrative  and  legal  suffi¬ 
ciency  before  issuance  by  the  PEB  president.  After  the  review,  the  service  mem- 


Disability  Evaluation  System  Overview  39 


ber  later  receives  the  final  Formal  PEB  findings  and  the  rationale  for  the  findings 
from  the  PEB  president  via  certified  mail.  A  service  member  may  choose  to  ac¬ 
cept  the  Formal  PEB  findings  or  submit  a  Petition  for  Relief  to  the  Director,  Naval 
Council  of  Personnel  Board,  who  is  the  next  level  of  appeal.  The  member  has  15 
calendar  days  from  the  date  of  receipt  of  the  Formal  PEB  findings  to  submit  a 
Petition  for  Relief.  If  the  service  member  accepts  the  findings  and  rec¬ 
ommendations,  the  case  is  finalized  and  the  PEB  issues  a  Notice  of  Decision  to 
the  Chief  of  Navy  Personnel  or  to  the  Commandant,  Marine  Corps  (Manpower 
and  Reserve  Affairs) . 

♦  When  the  Air  Force  Formal  PEB  members  reach  agreement  on  a  fitness  and  dis¬ 
position  recommendation,  they  call  the  service  member  and  the  member’s  at¬ 
torney  back  into  the  chambers  and  read  the  findings  and  recommendations,  at 
which  point  the  formal  board  concludes.  The  service  member  must  sign  a  docu¬ 
ment  stating  that  he  or  she  either  accepts  or  chooses  to  appeal  the  findings  and 
recommendations.  If  the  service  member  leaves  the  premises  without  signing  the 
document,  it  is  assumed  that  he  or  she  chooses  to  appeal. 

In  the  Departments  of  the  Navy  and  the  Air  Force,  the  PEB  approving  authority  re¬ 
views  the  Formal  PEB  findings  and  disposition  recommendations  for  every  medical 
board. 

Note  the  following  military  department-specific  difference: 

•  In  the  Army,  the  president  of  the  PEB  that  heard  the  case  reviews  the  board’s 
findings  and  recommendations.  However,  the  Army  Physical  Disability  Agency 
does  conduct  mandatory  reviews  of  the  cases  mentioned  earlier. 

DoD  Instruction  1332.38  establishes  the  following  time  requirement  goal  for  the 
physical  evaluation  and  disposition  phase,  including  the  appellate  review  and  dis¬ 
position  heyond  the  PEB  phase  (discussed  in  the  next  section)  of  the  DES:  “Upon  re¬ 
ceipt  of  the  [medical  board]  or  physical  evaluation  report  by  the  [Physical  Evaluation 
Board],  the  processing  time  to  the  date  of  the  final  renewing  authority  as  prescribed 
by  the  Secretary  of  the  Military  Department  should  normally  be  no  more  than  40 
days.” 

Phase  3.  Appellate  Review  and  Disposition  Beyond  the  Physical  Evaluation 
Botird 

By  law  (10  U.S.C.,  Ch.  61,  sec.  1214),  the  military  departments  may  not  separate  or 
retire  (for  disability)  a  service  member  without  a  full  and  fair  hearing  if  he  or  she 
demands  it.  The  Formal  PEB  meets  the  requirement  of  the  law.  However,  in  addition 
to  the  statutory  requirement,  the  military  departments  extend  two  to  three  additional 
appellate  review  opportunities  to  the  service  member.  Following  discharge  or 
permanent  retirement,  a  service  member  who  remains  dissatisfied  may  submit  a 
petition  to  the  appropriate  military  department’s  Board  of  Correction  of  Military 
Records. 
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Generally,  the  additional  appellate  review  boards  have  the  same  range  of  disposition 

options  as  the  PEBs. 

Note  the  following  military  department-specific  differences: 

•  The  Army  provides  three  levels  of  appellate  review  beyond  the  PEB;  the  last  two 
levels  are  components  of  the  Army  Council  of  Review  Boards. 

-  Army  Physical  Disability  Agency 

-  Army  Physical  Disability  Appeal  Board 

-  Army  Disability  Rating  Review  Board. 

The  Army  Physical  Disability  Agency  reviews  those  cases  in  which  the  service 
member  disagrees  with  the  findings  of  the  Formal  PEB  and  submits  a  rebuttal.  If 
the  agency  changes  the  findings  of  the  Formal  PEB  and  the  service  member  non¬ 
concurs  with  a  rebuttal,  tbe  case  is  forwarded  to  the  Army  Physical  Disability 
Appeal  Board  for  final  decision.  The  Army  Physical  Disability  Appeal  Board  re¬ 
views  disability  evaluation  cases  forwarded  by  the  Commanding  General,  Army 
Physical  Disability  Agency.  The  Army  Disability  Rating  Review  Board  reviews  dis¬ 
ability  percentage  ratings  at  the  request  of  a  service  member  who  was  retired  be¬ 
cause  of  physical  disability. 

•  The  Department  of  the  Navy  provides  two  levels  of  appellate  review  beyond  the 
PEB: 

-  Director,  Naval  Council  of  Personnel  Boards 

-  Officer  Disability  Review  Board. 

A  Department  of  the  Navy  service  member  who  disagrees  with  the  findings  and 
recommendations  of  the  Formal  PEB  may  submit  a  PFR  to  tbe  next  level  of 
appellate  review  beyond  the  Formal  PEB,  the  Director,  Naval  Council  of 
Personnel  Boards.  The  Director,  Naval  Council  of  Personnel  Boards  cannot 
reduce  the  final  disability  rating  assigned  by  the  Formal  PEB  unless  the  member 
is  offered  an  additional  appearance  before  a  Formal  PEB  whose  members  have 
not  previously  ruled  on  the  case  (U.S.  Department  of  the  Navy,  1998,  p.  5-2). 

The  Officer  Disability  Review  Board  reviews  a  limited  class  of  disability  cases 
wherein  officers  were  retired  or  released  from  active  duty  without  pay  for  physi¬ 
cal  disability  (U.S.  Department  of  the  Navy,  1998,  p.  7-1). 

•  The  Air  Force  provides  two  levels  of  appellate  review  beyond  the  PEB;  both  are 
components  of  the  Air  Force  Personnel  Council  (AFPC): 

-  Air  Force  Personnel  Board  (AFPB) 

-  Physical  Disability  Appeals  Board  (PDAB) . 

An  Air  Force  service  member  wbo  remains  dissatisfied  with  the  Formal  PEB 
findings  and  recommendations  may  appeal  his  or  her  case  to  the  AFPB,  which 
consists  of  five  senior  officers,  at  least  one  of  which  is  a  medical  officer.  The  AFPB 
conducts  a  records  review  of  the  case  in  closed  session.  The  board  members  re- 
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view  all  material  examined  by  the  Formal  PEB,  the  PEB  report,  and  the  service 
member’s  rebuttal.  The  physician  on  the  board  is  the  service  member’s  advocate. 
The  physician  reads  all  of  the  information  in  the  case  file  and  presents  a  sum¬ 
mary  to  the  rest  of  the  board.  He  or  she  answers  the  board’s  questions,  but  makes 
no  decisions  or  recommendations.  The  rest  of  the  members  vote  secretly: 
majority  vote  rules.  The  AFPB  may  change  the  findings  of  the  Formal  PEB. 

If  the  AFPB  agrees  with  the  Formal  PEB  or  grants  the  service  member’s  appeal, 
the  Secretary  of  the  AFPC  finalizes  the  case.  Any  other  major  change  results  in 
revised  findings  and  recommendations,  which  the  service  member  may  choose 
to  appeal  at  one  final  level,  the  PDAB.i^  This  board  consists  of  five  senior  officers, 
including  at  least  two  medical  officers,  and  conducts  a  records  review  in  closed 
session.  One  officer  briefs  the  case  to  the  other  members.  They  consider  the 
entire  medical  board  plus  the  service  member’s  rebuttal.  The  PDAB  issues 
findings  from  the  same  range  as  all  of  the  preceding  boards.  Majority  vote  rules. 
The  PDAB’s  decision  on  the  case  is  final. 


Phase  4.  Final  Disposition  by  the  Appropriate  Personnel  Authorities 

After  the  PEB  or  another  appellate  review  board  makes  the  final  disposition  decision, 
the  personnel  community  returns  the  service  member  to  duty  or  outprocesses  and 
issues  orders  for  those  separated  or  retired  for  disability. 

Note  the  following  military  department-specific  differences: 

♦  The  Total  Army  Personnel  Command  makes  final  disposition  of  disability  cases. 
The  Physical  Disability  Branch  within  the  Physical  Disability  Agency  calculates 
separation  and  retirement  dates  and  generates  orders. 

•  Navy  Personnel  Command  bandies  the  final  administrative  discharge  of  disabil¬ 
ity  cases.  The  effective  date  of  retirement  or  separation  because  of  physical  dis¬ 
ability  (either  permanent  or  temporary)  is  normally  within  four  to  six  weeks,  on 
average,  after  issuance  of  the  “Notification  of  Decision.”  The  four-  to  six-week 
elapsed-time  standard,  however,  is  a  guideline  and  not  an  inflexible  rule.  It  may 
be  exceeded  by  the  Chief  of  Naval  Personnel  or  the  Commandant  of  the  Marine 
Corps — Manpower  Management  Division,  Separation  and  Retirement  Branch, 
Disability  Separation  and  Retirement  Section  in  circumstances  such  as  when 
there  is  a  severe  hardship  on  the  member;  when  the  member  who  is  unable  to 
sell  earned  leave  takes  the  earned  leave  in  lieu  of  selling  it;  infeasibility,  such  as 
when  there  is  longer  lead  time  for  properly  vacating  government  quarters  or 
arranging  movement  of  household  effects;  and  adverse  effect  on  the  service  such 
as  when  the  four-  to  six-week  standard  precludes  contact  relief  of  officers  in 


^^The  Department  of  the  Air  Force  is  looking  at  doing  away  with  this  last  level  of  appeal.  It  processed 
about  a  dozen  cases  at  the  Physical  Disability  Appeals  Board  level  in  1998  and  no  findings  and 
recommendations  changed.  Senior  Air  Force  primary  participants  note  that  this  level  of  review  does  not 
seem  to  add  any  value  and  it  slows  down  case  processing  by  a  month. 
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command  or  other  key  billets  (Secretary  of  the  Navy  Instruction  1850.4D,  1998  p 
Ml). 

•  The  Secretary  of  the  Air  Force  Personnel  Council  (SAFPC)  handles  the  final  dis¬ 
position  of  each  disability  case.  The  Director,  SAFPC,  is  empowered  to  determine 
appropriate  disposition  and  announce  the  final  decision  of  the  Secretary.  In  turn, 
the  SAFPC  has  designated  certain  key  officials  in  the  U.S.  Air  Force  Physical 
Disability  Division  as  Special  Assistants  to  the  Director,  SAFPC.  These  assistants 
have  the  authority  to  finalize  cases  and  announce  the  final  secretarial  determi¬ 
nation  in  those  cases  not  otherwise  required  to  be  finalized  at  SAFPC  level. 

Setting  aside  the  stated  purpose  and  set  of  desired  system  outcomes  for  the  DES 
proposed  earlier  in  this  chapter,  Chapter  4  presents  our  issues-driven  analysis  of 
instances  of  variability  in  disability  policy  application  across  and  within  military  de¬ 
partments,  plus  recommended  interventions  to  achieve  more-consistent  policy  ap¬ 
plication.  Chapters  5  and  6  shift  from  the  issues-driven  approach  presented  in 
Chapter  4  back  to  the  recommended  purpose-driven  approach.  The  latter  approach 
relies  on  the  stated  purpose  and  set  of  desired  outcomes  proposed  in  this  chapter  in 
order  to  present  the  major  recommendations  of  this  report:  interventions  in  training 
and  management  information  system  deployment. 


Chapter  Four 


ISSUES  AND  INTERVENTIONS  FOR  ACHIEVING  CONSISTENT 

POLICY  APPLICATION 


In  Chapter  3,  we  propose  a  purpose  and  set  of  desired  outcomes  for  the  Disability 
Evaluation  System.  Chapters  5  and  6  fully  develop  this  top-down,  purpose-driven 
approach  to  conducting  the  DES  training  needs  analysis  that  lead  to  our  training  and 
management  information  system  recommendations.  This  chapter,  however, 
presents  our  bottom-up,  issues-driven  analysis  as  part  of  a  comprehensive  plan  to 
achieve  consistent  application  of  disability  policy  which,  likewise,  informed  our 
recommendations. 

Based  on  our  attending  the  military  departments’  major  training  events  and 
conducting  numerous  interviews  with  diverse  primary  participants^  in  the  course  of 
our  study,  we  identified  dozens  of  instances  of  variability  in  policy  application  across 
or  within  the  military  departments.  We  interviewed  policymakers  and  administrators 
from  both  the  personnel  and  medical  communities,  PEB  members,  and  attorneys 
from  all  three  military  departments.  We  also  spoke  informally  with  PEBLOs  and  pa¬ 
tient  administrators  at  PEBLO  workshops. 

We  captured  these  instances  of  variability  in  policy  application — as  well  as  problems 
identified  by  the  primary  participants — in  the  form  of  issues  to  be  resolved.  For  ex¬ 
ample,  three  instances  of  variation  in  policy  application  are  expressed  as  the  follow¬ 
ing  three  issues:  military  departments  describe  the  purpose  of  the  Disability 
Evaluation  System  differently;  no  Disability  Evaluation  System  process  owner  exists; 
and,  none  of  the  primary  participants  (except  the  PEBLOs  in  the  Department  of  the 
Navy  medical  centers)  in  the  medical  evaluation  phase  of  the  Disability  Evaluation 
System  work  for  the  0-6  who  oversees  the  Physical  Evaluation  Board.^  Appendix  C 
summarizes  the  complete  list  of  issues. 


^For  the  purposes  of  this  study,  we  identified  12  primary  participant  populations;  PEBLOs  and  disability 
evaluation  counselors;  patient  administrators;  physicians  at  MTFs;  Medical  Evaluation  Board  members  at 
MTFs;  Medical  Evaluation  Board  approving  authorities  at  MTFs;  PEB  administrative  action  officers;  PEB 
members;  PEB  approving  authorities;  appellate  review  board  members  beyond  the  formal  PEB;  active 
component  unit  commanders;  Reserve  component  commanders;  and  attorneys  who  represent  service 
members  during  appeals. 

^The  0-6s  who  are  assigned  to  oversee  the  military  departments’  PEBs  share  no  titles  in  common.  The 
Army  0-6’s  title  is  Deputy  Commander,  Army  Physical  Disability  Agency;  the  Department  of  the  Navy 
0-6's  title  is  President,  Physical  Evaluation  Board;  and  the  Air  Force  0-6’s  title  is,  Chief,  Air  Force  Physical 
Disability  Division.  For  convenience,  we  created  the  title  PEB  Approving  Authority  to  apply  to  all  three 
military  departments’  0-6s.  This  title  parallels  the  Medical  Evaluation  Board  Approving  Authority  and  it 
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This  chapter  describes  our  anaiysis  of  ail  the  issues  and  the  recommended  interven¬ 
tions  to  move  toward  more-consistent  application  of  disability  policy.  We  considered 
addressing  each  of  the  issues  individually;  however,  many  of  the  issues  are  interre¬ 
lated  and  others  require  interventions  that  are  common  across  more  than  one  issue. 
Consequently,  to  develop  a  comprehensive  plan  to  achieve  consistent  application  of 
disability  policy,  we  used  a  variation  of  goal  fabric  analysis. 

Our  application  of  goal  fabric  analysis  suggested  ten  broad  interventions,  each  con¬ 
sisting  of  specific  actions  for  resolving  the  particular  issues.  Because  the  recom¬ 
mended  interventions  are  based  on  reported  or  observed  instances  of  inconsistent 
policy  application— information  that  is  not  necessarily  complete,  objective,  or 
empirically  based— we  expect  that  the  interventions  are  not  as  finely  tuned  as  they 
otherwise  might  be. 

This  chapter  ends  with  introducing  a  shift  in  viewpoint — from  a  focus  on  ensuring 
consistent  policy  application  to  a  focus  on  improving  system  performance. 


GOAL  FABRIC  ANALYSIS  OVERVIEW 

Goal  fabric  analysis  is  a  “bottom-up”  planning  tool  for  identifying  actions  needed  to 
address  a  diverse  set  of  issues  and  organizing  those  actions  into  an  overall  plan.^  In 
other  words,  the  tool  is  well  suited  to  the  task  at  hand  in  this  study.  It  provides  a  con¬ 
text  within  which  to  thoroughly  identify  issues  and  necessary  actions  and  then  de¬ 
sign  a  comprehensive  plan  around  those  actions.  Goal  fabric  analysis  does  this  by 
tying  the  issues  to  the  desired  results  and  tying  the  results  to  both  the  specific  actions 
needed  to  bring  about  those  results  and  the  specific  organizational  objectives  and 
goals. 

As  a  prelude  to  employing  the  goal  fabric  analysis,  we  conducted  an  environmental 
assessment  by  recording  notes  on  how  the  primary  participants  describe  the  opera¬ 
tion  of  the  DBS.  This  assessment  highlighted  differences  in  how  the  primary  partici¬ 
pants  view  disability  policy  and  its  application,  how  well  the  primary  participants  are 
prepared  to  carry  out  their  responsibilities,  and  differences  in  the  problems  per¬ 
ceived  by  the  primary  participants.  We  recorded  each  of  their  differences  (for  exam¬ 
ple,  differences  in  the  statement  of  purpose  of  the  DBS  among  the  military  depart¬ 
ments  or  in  the  interpretation  of  standards  contained  in  the  DoD  Directive  or 
Instruction)  and  recorded  each  significant  problem  as  an  "issue."  The  issues  were  the 
starting  point  for  employing  the  goal  fabric  analysis  framework,  which  is  displayed  in 
Figure  4.1. 


allows  us  to  easily  differentiate  the  PEB  Approving  Authority  primary  participant  population  from  other 
primary  participant  populations  of  the  DES  when  we  focus  on  training  in  later  chapters, 

^To  learn  more  about  goal-fabric  analysis  see  Gulick  and  Kuskey  (n.d.). 
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Figure  4. 1 — Goal  Fabric  Analysis  Framework 


For  each  issue,  we  asked  the  same  question:  What  would  we  observe  (in  relation  to 
the  issue)  if  the  difference  were  eliminated  or  the  problem  were  solved?  We  called 
this  observation  the  “desired  result.”  We  arrived  at  a  desired  result  for  each  distinct 
issue. 

In  the  next  step,  we  asked  two  separate  questions  for  each  desired  result.  First,  what 
specific  actions  would  bring  about  the  desired  result?  We  identified  a  single  action 
for  some  results,  identified  several  parallel  or  serial  actions  for  other  results,  and 
posited  alternative  actions  to  achieve  yet  other  results  (which  we  evaluated  in  a  later 
stage).  Second,  we  asked,  if  the  desired  result  were  accomplished,  what  objective 
would  it  serve?  We  identified  eight  objectives  that  appear  to  span  the  desired  results. 
Multiple  desired  results  serve  each  objective,  and  some  desired  results  serve  multiple 
objectives.  This  multiplicity  of  interactions  is  why  this  framework  is  called  a  goal 
“fabric.” 

In  the  final  step  of  employing  the  framework,  we  asked,  if  the  individual  objectives 
were  achieved,  what  broad  organizational  goals  would  they  serve?  We  identified 
three  broad-based  goals.  These  goals  support  an  implicit  superordinate  goal  that  we 
state  simply  as:  Ensure  the  consistent  application  of  disability  policy  within  and 
across  the  military  departments  where  appropriate.  This  analysis  focused  on 
identifying  “desired  results”  and  the  actions  necessary  to  accomplish  the  desired 
results — in  other  words,  formulating  a  near-term  plan  of  action.  However, 
implementation  of  the  recommendations  based  on  this  analysis  must  include 
actions  to  identify  and  measure  “actual  results,”  as  illustrated  in  Figure  4.1.  Appendix 
D  presents  a  specific  example  of  the  goal  fabric  analysis  framework  development  and 
delineates  the  full  set  of  objectives  and  goals  the  analysis  evoked. 
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The  goal  fabric  analysis  process  may  appear  to  unfold  in  reverse  order,  presuming 
that  an  organization  should  start  with  a  goal  (the  top  of  most  strategic  planning 
frameworks)  and  work  “down”  through  objectives  and  desired  results  to  identify  ac¬ 
tions.  Generally,  we  would  agree  with  this  observation;  however,  we  found  no 
established  and  shared  set  of  objectives  or  goals  organization-wide  (or  even  a  shared 
statement  of  purpose)  within  the  OSD  or  military  departments.^  The  strength  of  the 
goal  fabric  framework  lies  precisely  in  its  capability  to  make  explicit  the  objectives 
and  goals  that  underlie  a  recommended  set  of  actions. 

Although  the  goal  fabric  analysis  starts  at  the  bottom  with  issues  and  Figure  4.1  sug¬ 
gests  the  process  is  unidirectional,  it  is  in  fact  iterative.  The  process  begins  with  iden¬ 
tifying  issues,  then  in  turn  formulating  desired  results,  actions,  objectives,  and  goals. 
When  the  goals  have  been  formulated  based  on  a  bottom-up  analysis,  tbe  process 
begins  to  iterate,  starting  at  the  "top”  with  each  goal  identified  and  then  asking  (1) 
whether  the  goal  would  be  accomplished  if  the  supporting  objectives  were  success¬ 
fully  achieved;  (2)  whether  all  the  necessary  objectives  were  identified;  and  (3) 
whether  all  identified  objectives  were  necessary  for  accomplishing  the  goals.  The  it¬ 
eration  continues,  asking  (1)  whether  each  objective  would  be  accomplished  if  the 
supporting  desired  results  were  obtained;  (2)  whether  all  the  necessary  results  were 
identified;  and  (3)  whether  all  identified  results  were  necessary.  This  iterative  proce¬ 
dure  results  in  a  more  robust  set  of  desired  results,  objectives,  and  goals  in  which  to 
organize  the  necessary  actions. 

The  product  of  this  goal  fabric  analysis  comprises  ten  categories  of  interventions 
(each  composed  of  similar  actions)  together  with  assignment  of  responsibility.  The 
analysis  is  couched  in  terms  of  the  goals  and  objectives  the  actions  are  designed  to 
achieve.  Through  this  iterative  process,  a  goal  fabric  analysis  evokes  the  plan’s  overall 
goal  (in  this  case,  to  ensure  the  consistent  application  of  disability  policy  within  and 
across  the  military  departments  where  appropriate)  and  links  the  goal  to  the  many 
actions  necessary  to  achieve  it.  In  the  same  way,  the  analysis  prioritizes  the  necessary 
actions  and  their  desired  results  in  the  larger  context  of  the  objectives  they  are  in¬ 
tended  to  serve.  In  effect,  the  goal  fabric  analysis  produces  a  near-term  plan  that 
management  uses  to  ensure  that  the  interventions  are  carried  out. 

Finally,  with  this  near-term  plan  in  place,  the  OSD  can  monitor  the  plan’s  implemen¬ 
tation  by  focusing  on  the  actual  results  of  the  actions  taken.  This  is  conveyed  by  the 
loop  shown  on  the  right  side  of  Figure  4.1. 

APPLYING  GOAL  FABRIC  ANALYSIS  TO  RESOLVE 
IDENTIFIED  ISSUES 

To  apply  goal  fabric  analysis  to  the  issues  at  hand,  we  employed  a  spreadsheet  that 
(1)  linked  issues  to  desired  results;  (2)  linked  desired  results  to  both  actions  to  bring 
about  those  results  and  the  objective(s)  the  results  support;  and  (3)  linked  objectives 


^In  fact,  as  discussed  in  the  final  section  of  this  chapter,  we  employ  just  such  a  top-down  approach,  using 
the  purpose  and  set  of  outcomes  proposed  in  Chapter  3,  to  develop  recommendations  for  training  and  a 
management  information  system. 
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to  the  overall  goals  they  support.  This  goal  fabric  model  allowed  us  to  group  the 
actions  together  in  different  ways,  while  always  retaining  the  link  to  the  issues  from 
which  they  originated  and  the  higher  order  they  serve. 

Ten  categories  of  interventions  evolved  as  the  most  useful  means  of  resolving  the 
identified  issues.  Each  intervention  category,  as  follows,  contains  similar  types  of  ac¬ 
tions,  many  of  which  build  on  other  actions  in  the  same  or  different  categories: 

•  Assistant  Secretary  of  Defense  Decisions 

•  Policy  Guidance 

•  Organizational  Changes 

•  Personnel  Policy 

•  Personnel  Management 

•  Training 

•  Information  Source  Development 

•  Management  Information  System  Deployment 

•  Process 

•  Incentives. 

These  intervention  categories  cut  across  the  various  phases  of  the  DBS,  the  primary 
participant  populations,  and  the  objectives  and  goals  evoked  through  the  goal  fabric 
analysis.  Nevertheless,  we  found  that  organizing  the  necessary  actions  into  ten  inter¬ 
vention  categories  was  the  best  means  to  present  a  comprehensive  plan  to  the  OSD. 
The  OSD  could  thereby  use  the  plan  to  move  toward  a  more  consistent  application  of 
disability  policy  based  on  the  instances  (that  is,  the  issues)  that  exist  today. 

The  issues-driven  goal  fabric  analysis  reinforced  the  importance  of  training  DES 
participants— the  genesis  of  this  study— and  management  information  system  de¬ 
ployment  as  key  interventions  to  ensure  consistent  application  of  disability  policy 
across  and  within  military  departments  (the  management  information  system  is  dis¬ 
cussed  in  Chapter  6).  A  separate  purpose-driven  analysis  indicated  that  these  two 
interventions  are  also  keys  to  improving  overall  system  performance. 

This  study  also  called  for  developing  a  process  to  monitor  the  effectiveness  of  the 
changes  in  training  and  other  interventions,  which  led  to  the  recommended  man¬ 
agement  information  system.  Chapters  5  and  6  discuss  the  training  and  management 
information  system  deployment  interventions  in  more  depth.  These  two  interven¬ 
tions  are  the  most  resource  intensive  of  the  ten  intervention  categories,  which  are 
covered  in  the  following  sections,  and  offer  the  greatest  prospects  for  increasing 
overall  system  performance. 

Appendix  E  groups  the  actions  by  intervention  categories  and  by  the  objectives  they 
support. 
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Assistant  Secretary  of  Defense  Decisions  Intervention 

The  Assistant  Secretary  of  Defense  Decisions  intervention  focuses  on  two  types  of 
ASD  decisions:  (1)  those  that  result  in  a  common  understanding  of  the  purpose  of  the 
DES  and  (2)  those  that  result  in  a  common  understanding  of  the  standards  for  op¬ 
erationalizing  disability  policy.  With  regard  to  the  first  type  of  decision,  the  variations 
in  the  operational  characteristics  of  the  DES  across  the  military  departments,  and  the 
differences  in  the  primary  participants’  observations  on  these  variations,  stem  in 
large  part  fi-om  varying  perspectives  on  the  purpose  of  the  DES. 

A  common,  shared,  and  clearly  articulated  statement  of  the  purpose  of  the  DES  is 
critical  to  the  consistent  application  of  disability  policy.  Consequently,  we  recom¬ 
mend  that,  as  the  first  intervention,  the  ASD/FMP,  in  coordination  with  the  ASD/RA 
and  the  ASD/HA,  decide  on  an  explicit  statement  of  the  purpose  of  the  DES.  The 
ASD/FMP  should  direct  the  Director,  Officer  and  Enlisted  Personnel  Management,  to 
consult  with  a  small  group  of  experienced  DES  experts  representing  the  military  de¬ 
partments’  PEBs  and  Office  of  the  Surgeons  General  to  produce  recommendations 
upon  which  the  three  ASDs  can  make  a  decision. 

In  addition,  we  found  that  primary  participants  in  the  medical  evaluation  phase  and 
physical  disability  evaluation  phase  of  the  DES  perceived  major  problems  with  each 
other’s  phase  of  the  system  (see  the  issues  in  Appendix  C).  We  believe  these 
perceptions  stem  from  a  mutual  lack  of  understanding  of  the  purpose  and  role  of  the 
Medical  Evaluation  Board  despite  the  changes  and  information  in  DoD  Instructions 
1332.38  and  1332.39.  Consequently,  we  recommend  that  the  ASD/FMP,  in 
coordination  with  the  ASD/RA  and  the  ASD/HA,  decide  on  a  statement  of  purpose 
for  the  Medical  Evaluation  Board.  The  same  small  group  of  experienced  DES  experts 
representing  the  military  departments’  PEBs  and  the  Office  of  the  Surgeons  General 
should  also  produce  recommendations  upon  which  the  three  ASDs  can  decide  upon 
a  mutually  acceptable  statement  of  the  purpose  of  the  Medical  Evaluation  Board  (not 
to  be  confused  with  the  medical  board  case  file)  within  the  overall  process. 

These  two  ASD  decisions  are  critical;  they  must  be  made  first  because  they  inform  all 
of  the  other  ASD  decisions  that  follow.  Although  we  would  have  preferred  to  recom¬ 
mend  specific  actions  in  all  ten  categories  of  interventions,  we  did  not  do  so  because 
without  a  clearly  defined  and  mutually  understood  DES  purpose  statement,  no  ef¬ 
fective  criteria  exist  to  choose  among  alternative  recommendations.^ 

With  regard  to  the  second  type  of  action  stemming  from  this  category  of  interven¬ 
tion— ASD  decisions  that  result  in  a  common  understanding  of  the  standards  for 
operationalizing  disability  policy — primary  participants  cited  numerous  examples  in 
which  primary  participants  in  the  DES  received  little  or  no  guidance,  or  ambiguous 
instructions,  regarding  the  specific  standards  to  employ  despite  the  changes  and 
information  in  DoD  Instructions  1332.38  and  1332.39.  The  first  eight  issues  in 


Although  we  propose  a  specific  stated  purpose  for  the  DES  in  Chapter  3  in  order  to  present  a 
methodology  for  developing  the  training  intervention  and  set  of  metrics  for  use  in  a  management 
information  system,  we  believe  the  DoD  itself  should  apply  that  methodology  to  the  DES  purpose 
Statement  the  three  ASDs  decide  upon.  ^ 
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Appendix  C  form  the  basis  for  these  recommendations.  For  the  Medical  Evaluation 
Board,  the  issues  fell  in  two  areas:  the  standards  for  referring  medical  boards  to  the 
PEBs  and  time  frames  for  initiating  Medical  Evaluation  Boards. 

We  recommend  that  the  ASD/FMP,  in  coordination  with  the  ASD/RA  and  the 
ASD/HA,  decide  on  appropriate  standards  for  referring  medical  boards  to  the  PEB 
and  appropriate  time  frames  for  initiating  Medical  Evaluation  Boards.  The  standards 
for  referring  medical  boards  to  the  PEB  should  allow  for  variations  among  military 
departments  based  on  their  different  missions  and  requirements;  however,  these  al¬ 
lowable  variations  and  the  reasons  for  them  should  be  clearly  enunciated.  The 
ASD/FMP  should  direct  the  Director,  Officer  and  Enlisted  Personnel  Management,  to 
consult  with  a  small  group  of  experienced  DES  experts  representing  the  military  de¬ 
partments’  PEBs  and  the  Office  of  the  Surgeons  General  to  produce  recommenda¬ 
tions  upon  which  the  three  ASDs  can  make  the  decision. 

In  a  somewhat  different  context,  we  recommend  that  the  ASD/FMP,  in  coordination 
with  the  ASD/RA  and  the  ASD/HA,  decide  on  mechanisms  for  seamless  transmission 
of  medical  boards  from  one  military  department  to  another.  These  mechanisms 
should  result  in  data  that  is  needed  and  formatted  to  expeditiously  incorporate  a 
medical  board  from  one  military  department  into  the  PEB  of  another.  The  ASD/FPM 
should  direct  the  Director,  Officer  and  Enlisted  Personnel  Management,  to  consult 
with  a  small  group  of  experienced  DES  experts  representing  the  military  depart¬ 
ments’  PEBs  and  the  Office  of  the  Surgeons  General  to  produce  recommendations 
upon  which  the  three  ASDs  can  make  the  decision. 

Numerous  primary  participants  in  the  DES  expressed  confusion  and  frustration  be¬ 
cause  they  receive  little  or  no  guidance,  or  ambiguous  instructions,  regarding  the 
specific  standards  to  employ  despite  the  changes  and  information  in  DoD 
Instructions  1332.38  and  1332.39.  For  the  PEBs,  the  issues  covered  four  areas:  (1)  the 
reasons  for  nondeployability,  and  the  use  of  nondeployability  in  determinations  of 
fitness;  (2)  more  broadly,  the  standards  for  determining  fitness;  (3)  aspects  of  the  in¬ 
formation  used  by  the  PEB  to  determine  fitness  and  disability  ratings;  and  (4)  the 
amount  of  time  authorized  to  a  service  member  to  make  an  election  following  a  PEB 
decision. 

We  recommend  that  the  ASD/FMP,  in  coordination  with  the  ASD/RA  and  the 
ASD/HA,  decide  on  appropriate  standards  for  determining  fitness;  the  information 
the  PEB  should  use  for  determining  fitness  and  disability  rating;  and  a  consistent  pe¬ 
riod  of  time  among  the  services  to  allow  the  service  member  to  elect  options  follow¬ 
ing  a  PEB  decision.  The  ASD/FMP  should  direct  the  Director,  Officer  and  Enlisted 
Personnel  Management,  to  consult  with  a  small  group  of  experienced  DES  experts 
representing  the  military  departments’  PEBs  and  the  Office  of  the  Surgeons  General 
to  produce  recommendations  upon  which  the  three  ASDs  can  make  a  decision. 

As  a  prelude  to  making  recommendations,  the  small  group  of  DES  experts  should  ex¬ 
amine  and  determine  appropriate  criteria  for  nondeployability  and  use  of  nonde¬ 
ployability  in  determinations  of  fitness.  The  standard  agreed  upon  should  accom¬ 
modate  variations  among  military  departments  based  on  their  different  missions  and 
requirements;  however,  the  standard  should  clearly  enunciate  the  allowable  varia- 
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tions  and  the  reasons  for  them.  The  group  should  consider  and  agree  upon  the  type 
of  information  the  PEB  should  use  to  determine  fitness  and  disability  ratings.  The 
small  group  of  DES  experts  should  also  determine,  across  the  military  departments,  a 
consistent  period  of  time  to  make  an  election  following  a  Physical  Evaluation  Board 
decision  or  explain  how  differences  would  still  allow  for  due  process. 

To  summarize  the  Assistant  Secretary  of  Defense  Decisions  intervention; 

•  The  ASD/FMP,  direct  the  Director,  Officer,  and  Enlisted  Personnel  Management 
to  consult  with  a  small  group  of  experienced  DES  experts  representing  the  mili¬ 
tary  departments’  PEBs  and  the  Office  of  the  Surgeons  General  to  produce  rec¬ 
ommendations  upon  which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
the  ASD/RA,  can  decide  upon 

-  a  statement  of  the  purpose  and  the  DoD’s  desired  outcomes  of  the  DES 

-  a  statement  of  the  purpose  of  the  Medical  Evaluation  Board  within  the  overall 
process 

-  appropriate  time  frames  for  initiating  Medical  Evaluation  Boards 

-  appropriate  standards  for  referring  medical  boards  to  the  PEB 

-  mechanisms  for  seamless  transmission  of  medical  board  information  from  one 
military  department  to  another 

-  appropriate  standards  for  determining  fitness 

-  the  information  the  PEB  should  use  to  determine  fitness  and  disability  rating 

-  a  consistent  period  of  time  among  the  services  to  allow  for  service  member  elec¬ 
tion  of  options  following  a  PEB  (or  higher-level  appellate  review  board)  decision. 


Policy  Guidance  Intervention 

The  policy  guidance  intervention  focuses  on  two  types  of  actions:  (1)  formalization  of 
the  ASD  decisions  recommended  in  the  preceding  section  and  (2)  specific  OSD  di¬ 
rection  to  require  the  military  departments  to  use  expanded  certification  as  a  means 
of  ensuring  a  common  understanding  throughout  the  DES. 

DoD  Directive  1332.18  and  DoD  Instruction  1332.38  address  the  following  issues,  yet 
numerous  primary  participants  identified  the  issues  as  ongoing  problems  that  cause 
confusion  and  frustration.  Therefore,  with  regard  to  the  first  type  of  action,  we 
recommend  that  the  OSD  formalize  the  ASD  decisions  through  reissuance  of  DoD 
Directive  1332.18  (1996)  and  DoD  Instruction  1332.38  (1996).  The  DoD  Directive 
should  incorporate  a  clearly  stated  purpose  of  both  the  DES  and  the  Medical 
Evaluation  Board  within  the  larger  system.  The  DoD  Instruction  1332.38  should  set 
forth 

•  an  appropriate  time  frame  for  initiating  Medical  Evaluation  Boards 

•  clearly  stated  standards  for  referring  medical  boards  to  a  PEB 
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•  clearly  stated  standards  for  determining  fitness,  including  explicit  guidance  re¬ 
garding  the  role  of  nondeployability 

•  the  information  the  PEB  should  use  to  determine  fitness  and  disability  ratings 

•  a  definition  of  a  consistent  period  of  time  for  service  members  to  elect  options 
following  a  PEB  or  higher-level  appellate  board  decision. 

The  DoD  Instruction  1332.38  should  also  clearly  set  forth  stated  standards  for  the 
medical  board  format  and  the  minimum  information  needed  for  the  seamless 
transmission  of  medical  boards  from  one  military  department  to  the  PEB  of  another. 

In  addition,  the  military  departments  should  expeditiously  incorporate  these 
changes  into  (1)  their  instructions  and  regulations;  (2)  the  existing  training  programs 
and  those  that  we  propose  in  this  report;  and  (3)  the  information  sources  available  to 
all  primary  participants. 

With  regard  to  the  second  type  of  action,  we  recommend  inserting  a  broader  re¬ 
quirement  for  certification  than  the  requirement  contained  in  DoD  Directive  1332.18 
today.  The  current  DoD  Directive  requires  that  the  Secretaries  of  the  military  de¬ 
partments  “ensure  that  physicians  who  serve  on  MEBs  [medical  boards]  are  trained 
in  the  preparation  of  MEBs  [Medical  Evaluation  Boards]  for  physical  disability 
evaluation.”  The  DoD  Directive  also  requires  the  Secretaries  to  ensure  that  PEB 
members  and  applicable  review  authorities  are  trained  and  certified  in  disability 
evaluation.  We  found  no  mechanism  within  the  military  departments  on  which  the 
Secretaries  could  rely  in  order  to  ascertain  whether  they  were,  in  fact,  carrying  out 
this  direction.  Certification  is  an  excellent  means  of  ensuring  that  appropriate 
training  has  been  conducted  and  appropriate  information  sources  have  been  used.  It 
is  an  effective  means  of  ensuring  shared  understanding  of  the  DES  purpose  and 
desired  outcomes,  performance  time  frames,  and  performance  standards  throughout 
the  DES. 

We  recommend  certification  for  the  following  primary  participants:  (1)  PEBLOs  and 
disability  evaluation  counselors;  (2)  patient  administrators  who  support  the  DES  on  a 
regular  basis;  (3)  Medical  Evaluation  Board  approving  authorities;  (4)  PEB  members, 
approving  authorities,  and  administrative  action  officers;  (5)  physicians  who  write 
narrative  summaries  and  specialty  consultations,  and  those  who  serve  on  Medical 
Evaluation  Boards;  and  (6)  unit  commanders.  In  particular,  we  recommend  “self- 
certification”  for  physicians  who  write  narrative  summaries  and  specialty  consulta¬ 
tions  and  those  who  serve  on  Medical  Evaluation  Boards,  as  well  as  unit  comman¬ 
ders.  For  example,  the  medical  board  should  contain  a  statement  signed  by  the 
contributing  physicians  that  certifies  they  relied  on  available  training  and 
information  sources  in  preparing  their  input.  Likewise,  the  commander’s  letter 
should  indicate  whether  the  commander  used  available  training  and  information 
resources  in  developing  his  or  her  input.  This  information  should  be  collected  as  data 
in  the  management  information  system. 

Consequently,  the  OSD  should  strengthen  and  expand  leadership  direction  in  the 
DoDD  1332.18  and  DoDI  1332.38.  In  particular,  we  recommend  that  the  DoD 
Directive  require  training  and  certification  for  (1)  PEBLOs  and  disability  evaluation 
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counselors;  (2)  patient  administrators  who  support  the  DES  on  a  regular  basis;  (3) 
Medical  Evaluation  Board  approving  authorities;  (4)  PEB  members,  approving 
authorities,  and  administrative  action  officers;  (5)  physicians  who  write  narrative 
summaries  and  specialty  consultations,  and  those  who  serve  on  Medical  Evaluation 
Boards;  and  (6)  unit  commanders  who  submit  commander’s  letters. 

The  requirements  for  supporting  certification  are  discussed  briefly  later  in  this 
chapter  and  in  greater  detail  in  Chapter  5.  Chapter  6  highlights  the  need  for  the  rec¬ 
ommended  management  information  system  to  collect  data  on  the  certification 
status  of  primary  participants  as  a  key  measure  of  system  performance. 

To  summarize  the  Policy  Guidance  intervention: 

•  The  OSD — formalize  the  decisions  listed  in  the  previous  section  through  reis¬ 
suance  of  DoD  Directive  1332.18  and  DoD  Instruction  1332.38. 

•  The  DoD  Directive— require  expanded  use  of  certification  as  a  means  of  ensuring 
a  common  understanding  throughout  the  DES;  for  example,  training  and  certify¬ 
ing  physicians  who  dictate  narrative  summaries  and  write  specialty  consults,  unit 
commanders  who  submit  a  commander’s  letter, ^  and  PEBLOs,  as  well  as  Medical 
Evaluation  Board  approving  authorities  and  PEB  members,  approving  authori¬ 
ties,  and  administrative  action  officers. 


Organizational  Change  Intervention 

We  recommend  two  fundamental  organizational  changes  that  would  cut  across  the 
DES:  (1)  designation  of  a  process  owner  for  each  military  department  DES  and  (2) 
establishment  of  an  oversight  committee  at  the  Deputy  Assistant  Secretary  of 
Defense  level  to  assess  system  performance  and  expeditiously  resolve  issues  that  the 
Disability  Advisory  Council  cannot. 

A  number  of  primary  participants  frequently  cited  the  interface  between  the  Medical 
Evaluation  Board  and  the  PEB  as  a  particular  problem  source.  Those  charged  with 
carrying  out  the  medical  evaluation  phase  perceive  the  PEB  to  be  undervaluing  the 
medical  assessment  of  the  service  member’s  ability  to  perform  his  or  her  duties. 
Those  charged  with  carrying  out  the  physical  evaluation  phase  perceive  the  physi¬ 
cians  to  be  providing  incomplete  or  inaccurate  information  upon  which  the  PEB 
must  make  its  determinations.  Both  perceptions  are  correct. 

We  recommend  specific  actions  in  other  categories  of  interventions  that  will 
ameliorate  many  of  the  current  problems.  However,  a  more  fundamental  problem 
results  from  a  lack  of  accountability  for  the  overall  process,  which  currently  resides 
only  at  the  level  of  Secretary.  As  a  result,  we  recommend  that  the  Secretaries  of  the 
military  departments  designate  a  process  owner  who  is  responsible  for  oversight  and 


6 

Unit  commanders  must  sign  a  document  that  describes  the  impact  of  the  service  member’s  medical 
condition  on  the  member’s  ability  to  perform  his  or  her  normal  military  duties  and  to  deploy  or  mobilize, 
as  applicable.  This  document  is  commonly  referred  to  as  the  “commander's  letter.’’ 
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control  of  the  overall  operation  of  the  DBS  and  is  accountable  for  all  outcomes  within 
each  military  department's  DBS. 

Within  the  OSD,  as  opposed  to  the  individual  military  departments,  oversight  for  the 
DBS  resides  with  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness,  who  is 
the  overall  process  “owner."  However,  no  formal  forum  exists  beyond  the  Disability 
Advisory  Council,^  which  is  composed  of  0-5s,  0-6s,  and  GS-15s,  to  provide 
oversight  of  the  DBS,  to  examine  problems  and  make  decisions  that  the  Disability 
Advisory  Council  cannot  efficiently  resolve  because  of  the  composition  (organiza¬ 
tional  level)  of  its  membership,  and  to  evaluate  overall  DBS  performance. 

We  recommend  that  the  Under  Secretary  form  a  standing  committee— to  be  called  a 
Disability  Bvaluation  Committee — at  the  Deputy  Assistant  Secretary  of  Defense  level. 
Membership  should  include  the  Deputy  Assistant  Secretary  of  Defense  for  Military 
Personnel  Policy  (the  chair),  the  appropriate  Deputy  Assistant  Secretaries  represent¬ 
ing  the  Assistant  Secretaries  for  Health  Affairs  and  Reserve  Affairs,  and  an  appropri¬ 
ate  Deputy  Assistant  Secretary  from  each  military  department. 

The  current  Disability  Advisory  Council  would  bring  unresolved  issues  to  the 
Disability  Bvaluation  Committee  during  quarterly  or  biennial  meetings.^  In  addition, 
the  committee  should  review  and  evaluate,  at  least  annually,  information  from  the 
management  information  system,  which  is  discussed  again  later  in  this  chapter  and 
described  more  fully  in  Chapter  6.  The  committee  should  direct  DBS  actions  to  the 
Disability  Advisory  Council  and  the  military  departments,  as  appropriate,  based  on 
its  analysis  of  that  information.  More  importantly,  establishing  a  Disability 
Bvaluation  Committee  will  raise  the  visibility  of  the  DBS  within  the  DoD. 

To  summarize  the  Organizational  Change  intervention: 

•  The  Secretary  of  each  military  department — designate  a  process  owner^  for  the 
department's  DBS. 


^According  to  the  Disability  Advisory  Council  charter,  April  28,  1998,  “The  Disability  Council  will  be 
chaired  by  the  Office  of  the  deputy  Assistant  Secretary  of  Defense  MPP  (Officer  and  Enlisted  Personnel 
Management)  Director  or  their  designee.  The  Assistant  Secretaries  of  Defense  (Health  Affairs)  and 
(Reserve  Affairs)  will  nominate  representatives  to  serve  on  the  Disability  Council.  The  Secretaries  of  the 
Military  Departments  shall  also  appoint  representatives.  The  specific  representatives  may  be  chosen  at  the 
discretion  of  the  Secretaries  of  the  Military  Departments.  Normally,  the  Secretary  of  the  Army  shall 
appoint  the  Deputy  Commander,  U.S.  Army  Physical  Disability  Agency,  and  a  representative  of  the  Office 
of  the  Army  Surgeon  General.  Normally,  the  Secretary  of  the  Navy  shall  appoint  the  Director,  Naval 
Council  of  Personnel  Boards,  and  a  member  of  the  Office  of  the  Chief  of  the  Bureau  of  Medicine  and 
Surgery.  Normally,  the  Secretary  of  the  Air  Force  shall  appoint  the  Chief,  USAF  [U.S.  Air  Force]  Disability 
Division,  and  a  representative  of  the  Office  of  the  Air  Force  Surgeon  General.  The  Office  of  General 
Counsel,  Department  of  Defense,  shall  designate  the  legal  advisor  to  the  Disability  Council.  The  Secretary 
of  the  Department  of  Veterans  Affairs  shall  provide  representation  from  the  Office  of  the  Under  Secretary  for 
Benefits  [sic].” 

^The  structure  could  be  modeled  on  the  DoD's  Per  Diem,  Travel,  and  Transportation  Allowance 
Committee  (PDTTAC),  which  is  composed  of  membership  at  the  Deputy  Assistant  Secretary  level  and 
supported  by  a  military  and  civilian  advisory  panel  at  the  field  grade  and  civilian- equivalent  level.  We  do 
not  recommend,  however,  the  addition  of  any  staff  for  the  Disability  Advisory  Council  (other  than  the 
personnel  that  currently  serve),  as  suggested  by  the  PDTTAC  model. 

^Kprocess  owner  an  individual  or  team  designated  for  oversight  of,  control  of,  and  accountability  for  all 
activities  constituting  a  complete  process — in  this  case,  each  military  department’s  DES. 
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•  The  Under  Secretary  of  Defense  for  Personnel  and  Readiness — establish  a  senior 
leadership  oversight  committee  at  the  Deputy  Assistant  Secretary  of  Defense 
level  to  assess  system  performance  and  to  resolve  issues  that  the  Disability 
Advisory  Council  cannot  resolve  expeditiously. 

Personnel  Policy  Intervention 

We  recommend  personnel  policy  intervention  in  two  areas:  (1)  personnel  policies 
that  directly  affect  personnel  in  the  DES  and  (2)  personnel  policies  that  potentially 
affect  how  primary  participants  in  the  DES  make  their  decisions. 

Regarding  the  first  intervention  area,  the  process  owners  (as  recommended  in  the 
previous  section)  need  not  actually  control  all  the  resources  of  the  military  depart¬ 
ment  DES.  However,  if  they  do  not,  we  recommend  that  they  nevertheless  assess  the 
performance  of  the  military  department  PER  approving  authority  and  the  MTF 
commanders.  The  process  owners  should  provide  their  assessment  to  the  official 
who  writes  individual  performance  evaluations  for  the  PER  approving  authorities 
and  the  MTF  commanders.  The  individual  performance  assessment  should  be  based 
largely  on  the  information  that  is  gathered  and  reported  by  the  management 
information  system  operator. 

Regarding  the  second  intervention  area,  two  related  personnel  policies  have  the  po¬ 
tential  to  introduce  unwarranted  variation  into  PER  decisions  regarding  fitness 
and/or  disability  ratings: 

(1)  Some  primary  participants  we  interviewed  say  that  some  PERs  “adjust”  the  fitness 
or  disability  ratings  of  a  service  member  with  a  relatively  minor,  but  unfitting 
disability  who  is  nearing  20  years  of  service  in  order  to  allow  the  service  member  to 
retire  for  years  of  service.  In  a  case  like  this,  the  “adjustment”  might  consist  of  desig¬ 
nating  what  would  normally  be  an  unfit  determination  as  a  fit  determination  and 
returning  the  service  member  to  active  duty  so  that  the  member  can  go  ahead  and 
retire  for  years  of  service.  An  adjustment  might  also  consist  of  raising  a  disability 
rating  that  would  normally  be  10  or  20  percent  to  30  percent,  thereby  allowing  the 
member  to  retire  for  disability  (and  draw  disability  retirement  compensation)  rather 
than  separate  for  disability  (with  no  disability  compensation).  We  recommend  that 
the  OSD  articulate  an  explicit  policy  regarding  service  members  in  this  situation. 

Strong  cultural  incentives  exist  to  take  care  of  fellow  service  members.  Other  person¬ 
nel  policies  potentially  contribute  to  the  pressure  to  find  nondeployable  service 
members  unfit,  although  DoD  Instruction  1332.38  limits  the  extent  to  which  the  PEE 
can  use  nondeployability  as  the  sole  basis  for  unfitness.  We  recommend  the  services 
assess  the  possibility  of  placing  service  members  who  are  fit  but  not  deployable  into 
units  that  can  utilize  their  skills  and  experience  without  unduly  hampering  unit  ef¬ 
fectiveness  and  the  effective  operation  of  the  service  personnel  system.  The  assess¬ 
ment  should  be  conducted  with  a  view  toward  ensuring  the  best  use  of  trained  re¬ 
sources. 
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(2)  The  Defense  Authorization  Act  of  1993  amended  10  U.S.C.  1142  to  require  that 
service  members  receive  pre-separation  counseling  no  later  than  90  days  prior  to 
separation.  We  found  indications  that  the  military  departments  direct  members  to 
begin  pre- separation  counseling  before  a  fitness  determination  has  been  rendered, 
in  compliance  with  the  Defense  Authorization  Act  of  1993. 

Referral  for  pre- separation  counseling  sends  a  pretty  strong  message  that  a  service 
member  will  likely  be  separated  or  retired  and  potentially  creates  false  expectations 
on  the  part  of  the  service  member  when  in  fact  that  service  member  may  be  found  fit 
and  subsequently  returned  to  duty.  Specific  OSD  guidance  is  needed  to  correctly  in¬ 
terpret  application  of  this  statute  in  regard  to  service  members  undergoing  disability 
evaluation.  We  recommend  that  the  Office  of  the  ASD/FMP  review  the  impact  of  the 
Defense  Authorization  Act  of  1993  amendment  to  10  U.S.  Code  1142  as  it  applies  to 
service  members  undergoing  disability  evaluation  and  articulate  an  explicit  policy  re¬ 
garding  service  members  in  this  situation. 

To  summarize  the  Personnel  Policy  intervention: 

•  Process  owners — ^assess  the  performance  of  the  military  department  PEB  approv¬ 
ing  authority  and  MTF  commanders. 

•  The  services — assess  the  difficulty  of  placing  service  members  who  are  fit  but  not 
deployable  into  units  that  can  utilize  their  skills  and  experience 

•  The  OSD — ^articulate  an  explicit  policy  with  regard  to  fitness  and  disability  ratings 
for  a  service  member  who  is  nearing  20  years  of  service. 

•  The  OSD — ^review  the  impact  of  the  Defense  Authorization  Act  of  1993  amend¬ 
ment  to  10  U.S.C.  1142  that  requires  providing  pre-separation  counseling  for 
service  members  no  later  than  90  days  before  separation,  as  it  applies  to  service 
members  undergoing  disability  evaluation. 

Personnel  Management  Intervention 

Many  dedicated  and  capable  people  staff  the  various  positions  within  the  DES.  Many 
of  the  primary  participants  that  we  interviewed  acknowledge  that  the  DES  does  not 
receive  top  priority  in  terms  of  selecting  and  assigning  people  with  the  competencies 
and  experience  who  best  match  the  job  requirements.  The  DES  is  not,  unfortunately, 
considered  a  career-enhancing  assignment  for  many  military  personnel.  Never¬ 
theless,  increased  experience  generally  leads  to  better  performance.  Consequently, 
we  recommend  that  the  services  review  personnel  policies  with  the  objective  of 
increasing  PEBLO  performance  competencies,  in  particular,  through  a  combination 
of  experience  and  training. 

Our  observation  of  junior  noncommissioned  officers  and  petty  officers  serving  as 
PEBLOs  heightened  our  concerns  that  the  level  of  maturity  needed  for  the  tough  job 
of  counseling  required  in  these  assignments  may  be  lacking.  As  a  result,  we  recom¬ 
mend  that  the  military  departments  monitor  the  grades  of  individuals  assigned  as 
PEBLOs  and  disability  evaluation  counselors  and  notify  the  OSD  when  service  mem- 
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bers  below  the  pay  grade  of  E-6  are  assigned  to  these  positions,  and  the  military  de¬ 
partment’s  rationale  for  the  assignment. 

In  addition,  PEB  leadership  is  critically  important  to  the  successful  overall  operation 
of  the  DES.  Consequently,  we  recommend  that  the  PEB  approving  authorities  serve 
for  a  minimum  of  five  years. 

To  summarize  the  Personnel  Management  intervention: 

•  The  services — review  personnel  policies  with  the  objective  of  increasing  PEBLO 
performance  capabilities  through  a  combination  of  experience  and  training. 

•  The  services— monitor  the  grades  of  individuals  assigned  as  PEBLOs  and  disabil¬ 
ity  evaluation  counselors,  and  notify  the  OSD  when  service  members  below  the 
pay  grade  of  E-6  are  assigned  to  these  positions,  together  with  the  rationale  for 
the  assignment. 

•  The  military  departments — assign  PEB  approving  authorities  for  a  minimum  of 
five  years. 

Training  Intervention 

As  discussed  throughout  this  report,  this  study  was  chartered  to  produce  recom¬ 
mended  changes  to  the  training  provided  to  primary  participants  of  the  DES  to 
ensure  more-consistent  application  of  disability  policy  across  and  within  military  de¬ 
partments.  This  section  presents  the  results  of  our  issues-driven  training  needs  as¬ 
sessment,  which  suggests  that  three  major  actions  need  to  be  taken  to  move  toward 
more-consistent  application  of  disability  policy: 

1.  The  Office  of  the  ASD/FMP — develop  and  deliver  training  designed  to  expedite 
medical  board  processing. 

2.  The  Disability  Advisory  Council — ^sponsor  annual  symposia  for  representatives  of 
all  primary  participant  populations  across  military  departments. 

3.  The  military  departments — conduct  annual  symposia  for  primary  participants 
within  the  departments. 

The  PEBLO  training  provided  by  all  three  military  departments  identified  the  same 
set  of  obstacles  (which  we  call  issues)  to  efficient  processing  of  medical  boards 
through  the  DES,  and  focused  on  resolving  those  issues.  Likewise,  independent  in¬ 
terviews  with  numerous  diverse  primary  participants  identified  the  same  set  of  issues 
and  produced  recommended  training  content  that  could  be  used  in  all  the  military 
departments  to  resolve  those  issues  and  result  in  more-consistent  application  of 
disability  policy.  The  primary  participants  identified  the  following  priority  training 
content  to  resolve  the  issues: 

•  Template  for  narrative  summary  (contents  and  format) 

•  Medical  board  contents 

•  Required  medical  data  in  sufficient  detail  to  enable  cases  to  be  adjudicated 
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•  Documentation  of  rationale  supporting  Medical  Evaluation  Board  decisions 

•  Commander’s  letter/ nonmedical  assessment 

•  Documentation  of  rationale  supporting  PEB  decisions 

•  Differences  between  DoD  and  VA  disability  systems. 

Table  4.1  presents  the  training  content,  including  subcategories,  by  primary 
participant  populations,  as  identified  by  primary  participants  who  were  interviewed. 
The  full  range  of  information  for  some  topics  appears  in  italics  under  those  topics. 
The  comprehensive  training  recommendation  presented  in  Chapter  5  includes  the 
training  content  and  targeted  primary  participant  populations  listed  in  Table  4.1, 
with  the  exception  of  the  service  member  population. The  next  section  of  this 
chapter  presents  a  recommended  action  to  address  the  issue  of  service  members’ 
confusion  in  distinguishing  between  VA  and  DoD  disability  systems. 

Throughout  this  study,  we  perceived  that  the  primary  DES  participants  do  not  neces¬ 
sarily  think  of  themselves  as  part  of  a  system  or  that  what  they  do  is  part  of  a  process. 
Although  a  segment  of  the  training  content  recommended  in  Chapter  5  focuses  on 
participants  developing  a  broad  perspective  of  their  role  within  the  overall  DES — that 
is,  a  system  perspective — periodic  workshops  or  symposia  could  augment  formal 
training.  When  conducted  with  the  specific  objective  of  enhancing  communication 
to  produce  more-consistent  application  of  disability  policy,  workshops  can  be  a 
powerful  training  delivery  method  for  fostering  a  broader  system  perspective. 

Consequently,  we  recommend  that  the  Disability  Advisory  Council  sponsor  an  an¬ 
nual  cross-military  department  symposium  at  which  representatives  of  all  appropri¬ 
ate  primary  participant  populations  can  present,  review,  and  analyze  military 
department  data;  propose  corrective  actions;  and  identify  best  practices.  Periodic 
attendance  at  these  symposia  should  be  a  requirement  for  continued  primary  par¬ 
ticipant  certification. 

We  also  recommend  that  the  military  departments  conduct  annual  symposia  at 
which  department  primary  participants  present,  review,  and  analyze  service  data; 
propose  corrective  actions;  and  identify  best  practices. 

To  summarize  the  Training  intervention: 

•  The  OSD/FMP — develop  and  deliver  training  designed  to  expedite  medical  board 
processing. 

•  The  Disability  Advisory  Council — ^sponsor  an  annual  cross-service  symposium. 

•  The  military  departments — conduct  annual  symposia  for  all  primary  participant 
populations  in  the  DES  to  present,  review,  and  analyze  military  department  data; 
propose  corrective  actions;  and  identify  best  practices. 


^^Primary  participants  noted  that  many  congressional  inquiries  result  from  service  members  not 
understanding  the  difference  between  the  DoD  Disability  Evaluation  System  and  the  Department  of 
Veterans  Affairs  Disability  Compensation  System  and  therefore  believing  they  have  been  treated  unfairly. 
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Information  Source  Development  Intervention 

The  information  source  intervention  focuses  on  three  types  of  OSD  actions:  (1)  de¬ 
velopment  of  up-to-date  and  readily  available  information  banks  shared  by  the  mili¬ 
tary  departments;  (2)  creation  of  a  virtual  communications  network  for  the  primary 
participants;  and  (3)  creation  of  an  instrument  for  conducting  customer  surveys. 
With  regard  to  the  first  action,  see  Chapter  5  for  a  description  of  our  recommenda¬ 
tion  that  the  OSD  devote  a  Web  site  to  self-directed  computer-based  distance  train¬ 
ing  for  disability  evaluation. 

We  envision  the  training  packages  including  comprehensive  samples  of  the  docu¬ 
ments  required  to  process  a  case  through  the  DBS.  In  particular,  the  unit  com¬ 
manders’  training  package  should  contain  examples  of  well-written  and  effective 
commander’s  letters.  Ideally,  the  unit  commander  should  use  a  Web -based  template 
for  on-line  transmission  to  the  PEBLO  or  patient  administrator  handling  a  specific 
case.  In  addition,  the  unit  commanders’  training  package  should  provide  all  the 
information  commanders  must  have  regarding  the  need  for  and  preparation  of  a 
LOD  determination,  including  examples  of  LOD  determinations.  The  unit  com¬ 
manders  must  also  have  a  means  of  transmitting  the  information  electronically. 

Likewise,  we  envision  a  similar  Web-based  information  source  that  provides  a  train¬ 
ing  package  for  physicians.  In  particular,  the  physicians’  training  package  that  would 
be  accessed  from  the  proposed  Web  site  should  include  examples  of  well-written  and 
effective  narrative  summaries  and  specialty  consults.  Ideally,  a  physician  would  ac¬ 
cess  an  electronic  template  to  write  narrative  summaries  and  provide  specialty  con¬ 
sult  input. 

The  electronic  format  is  intended  as  a  user-friendly  guide  to  narrative  summary  re¬ 
quirements  (such  as  tests  and  measures  required  for  a  complete  medical  board)  for 
all  diseases  and  injuries  in  general  and  the  five  specialties  that  make  up  the  majority 
of  consults  in  particular.  The  electronic  format  overcomes  a  problem  we  identified: 
Unlike  paper  documents,  it  is  an  information  source  that  physicians  cannot  take  with 
them  when  they  rotate  to  a  new  assignment,  so  it  will  be  there  for  the  next  physician 
who  needs  the  training.  This  format  also  invites  interaction  because  it  is  physically 
available  and  current,  it  makes  physicians’  jobs  easier,  and  because  physicians  know 
it  makes  their  jobs  easier,  they  will  come  to  rely  on  it. 

If  the  OSD  does  not  develop  the  recommended  self-directed  computer-based  dis¬ 
tance  training,  it  should  incorporate  the  information  described  earlier  pertaining  to 
the  unit  commanders’  and  physicians’  training  into  the  medical  instructions  or  di¬ 
rectives  pertaining  to  the  Medical  Evaluation  Board.  The  Office  of  the  Surgeons 
General  should  update  the  medical  policy  documents  to  match  the  OSD  and  military 
departments’  disability  policy  documents,  and  describe  the  appropriate  format  and 
content  of  medical  boards. 

To  supplement  the  training  packages  for  PEBLOs/ disability  evaluation  counselors 
and  patient  administrators,  we  recommend  organizing  structured  information  in  a 
centralized  location  on  a  Web  site  for  frequent  updating.  The  Web  site  should  con¬ 
tain  DoD  and  military  department  directives,  instructions,  and  regulations;  contact 
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information  for  cohorts  and  centrally  located  experts;  frequently  asked  questions  and 
their  answers;  and  other  up-to-date  information. 

As  part  of  this  centrally  located  Web  site  information  source,  or  as  a  separate  source, 
we  recommend  that  the  OSD  provide  individual  service  members  access  to  all  the 
information  they  need  to  understand  the  DES  and  their  rights  and  entitlements  un¬ 
der  it,  through  either  a  Web  site  and/or  a  published  document.  In  particular,  this 
Web  or  print  document  should  include  a  comprehensive  comparison  of  the  DoD 
Disability  Evaluation  System  and  the  VA  Disability  System.  The  material  developed 
for  the  proposed  computer-based  distance-training  packages  (described  in  Chapter 
5)  can  serve  as  the  basis  for  developing  this  information  source,  which  could  also 
contain  answers  to  frequently  asked  questions. 

We  also  recommend  that  the  OSD  develop  a  database  of  DES  best  practices.  The 
database  should  contain  data  collected  from  the  recommended  workshops  and  sym¬ 
posia  suggested  earlier  in  this  chapter  and  from  the  virtual  communications  network 
we  recommend  next. 

With  regard  to  the  virtual  communications  network,  we  recommend  that  the  OSD 
establish  a  separate  mailing  list  server’  ’  for  the  Medical  Evaluation  Board  approving 
authorities,  another  for  the  PEB  members  and  approving  authorities,  and  lastly,  one 
for  the  PEBLOs.  A  list  server  offers  an  effective  means  of  bringing  consistency  to 
disability  policy  application,  particularly  to  cases  that  arise  infrequently.  Because  the 
Army  and  Navy  PEBs  are  geographically  dispersed,  they  especially  would  benefit 
from  it.  Each  primary  participant  population’s  list  server  should  include  all  the 
military  departments  in  order  to  share  the  greatest  amount  of  information.  As  in  the 
case  of  the  Medical  Evaluation  Board  approving  authorities  and  the  PEB  members 
and  approving  authorities,  a  list  server  for  PEBLOs  serves  as  a  mechanism  for 
soliciting  advice  from  the  entire  PEBLO  knowledge  base. 

With  regard  to  the  third  information  source  development  action — creation  of  an  in¬ 
strument  for  conducting  customer  surveys— we  recommend  that  the  OSD  develop  a 
survey  instrument  to  measure  customer  satisfaction,  which  the  services  would  ad¬ 
minister  to  every  service  member  who  has  contact  with  the  DES,  including  those  who 
are  returned  to  duty.  A  survey  of  satisfaction  is,  admittedly,  a  lagging  indicator  of  DES 
performance.  Nevertheless,  it  is  an  important  measure  of  system  outcomes.  To  “get 
ahead  of  the  system  (that  is,  to  measure  the  determinants  of  customer  satisfaction 
before  customer  satisfaction  is  negatively  affected),  we  propose  a  comprehensive 
management  information  system,  which  is  discussed  next.  The  customer  satisfaction 
survey  is  an  important  component  of  such  a  management  information  system. 

To  summarize  the  Information  Source  Development  intervention: 


’’According  to  wvvw.pcwebopaedia.com,  a  list  server  is  a  “server  that  manages  mailing  lists  for  groups  of 
users.  Two  of  the  most  popular  e-mail  mailing  list  server  systems  for  the  Internet  are  LISTSERV  and 
Majordomo. 
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•  The  OSD— produce  electronic  media  that  include  a  comprehensive  sample  of  the 
documents  needed  to  process  a  case  through  the  DBS,  together  with  easy-to-use 
reference  documents. 

•  The  OSD — develop  a  brochure  and/or  Web  site  for  individuals  separated  or  re¬ 
tired  for  disability  that  describes  the  service  member’s  rights,  benefits,  and  enti¬ 
tlements  and  the  significance  and  consequences  of  the  determinations  reached, 
including  a  comprehensive  comparison  of  VA  and  DoD  disability  systems. 

•  The  OSD — develop  and  maintain  a  database  of  "best  practices”  in  the  DBS. 

•  The  OSD — establish  a  list  server  for  Medical  Bvaluation  Board  approving  au¬ 
thorities,  another  for  PEB  members  and  PEB  approving  authorities,  and  another 
for  PEBLOs. 

•  The  OSD — develop  a  survey  instrument  to  measure  customer  satisfaction  that 
the  military  departments  administer  to  every  service  member  who  has  contact 
with  the  DES,  including  those  returned  to  duty. 


Management  Information  System  Deployment  Intervention 

Currently,  no  central  structured  mechanism  exists  to  gather  data  across  military  de¬ 
partments  to  inform  actions  or  assess  how  well  the  DES  accomplishes  its  intended 
purpose  and  desired  outcomes.  A  comprehensive  management  information  system 
with  this  data-gathering  capability  would  be  a  key  intervention  enabling  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness,  the  Assistant  Secretaries  of 
Defense,  and  the  Secretaries  of  the  military  departments  to  carry  out  their  responsi¬ 
bilities  under  DoD  Directive  1332.18.  A  system  capable  of  monitoring  key  perfor¬ 
mance  measures  in  the  DES  would  also  provide  the  necessary  foundation  for  an  insti¬ 
tutional  mechanism  for  quality  control  and  quality  assurance, 

Therefore,  we  recommend  that  the  ASD/FMP,  after  consulting  on  the  information 
needs  of  the  ASD/HA  and  ASD/RA,  direct  the  Director,  Officer  and  Enlisted 
Personnel  Management,  to  develop  and  maintain  a  comprehensive  management 
information  system  capable  of  monitoring  DES  performance  measures  (as  they  apply 
to  active  and  Reserve  components).  Chapter  6  describes  such  a  system  in  more 
depth. 

Based  on  the  issues  we  observed  in  the  medical  evaluation  phase  of  the  DES,  the 
MTF  commanders  and  the  Surgeons  General  need  information  from  a  management 
information  system  such  as  the  one  being  proposed.  The  MTF  commander  should 
review  data  at  their  most  disaggregated  level.  In  particular,  the  commander  should 
examine  reports  on  medical  boards  returned  by  the  PEB — for  insufficient  data  or  for 
any  other  reason— broken  out  by  reason  for  return,  referring  physician,  PEBLO,  and 
unit  commander.  The  Surgeon  General  should  review  a  more-aggregate  form  of  the 


l^Chapter  3  describes,  within  the  context  of  the  overall  DES  operating  framework,  the  existing  measures 
the  OSD  requires  the  military  departments  to  report. 
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data  to  determine  if  particular  MTFs  stand  out,  either  as  performance  benchmarks  or 
problem  areas,  with  a  focus  on  the  timeliness  of  Medical  Evaluation  Boards, 

To  provide  these  aggregated  and  disaggregated  reports,  the  management  informa¬ 
tion  system  should  be  capable  of  tracking  medical  boards  from  dictation  of  the  nar¬ 
rative  summary  to  the  signature  of  the  MTF  commander.  The  form  of  the  reports  can 
vary  among  the  military  departments  provided  they  track  individual  medical  boards 
and  can  summarize  the  total  elapsed  time  from  dictation  of  the  narrative  summary  to 
the  commander's  sign-off. 

In  addition,  as  noted  earlier,  DoD  Directive  1332.18  holds  the  Secretaries  of  the  mili¬ 
tary  departments  accountable  for  ensuring  that  physicians  are  trained  and  that  PEB 
members  are  trained  and  certified.  Certification  is  a  key  output  measure*^  for  moni¬ 
toring  system  performance.  In  order  for  the  Secretaries  to  carry  out  these  responsi¬ 
bilities,  we  recommend  that  the  management  information  system  operator  report  on 
the  certification  status  of  the  primary  participants  of  the  DES  (we  recommend  some 
additional  certification  requirements  in  the  earlier  section  on  policy  guidance 
interventions). 

To  summarize  the  Management  Information  System  Deployment  intervention: 

•  The  ASD/FMP,  after  consulting  on  the  information  needs  of  the  ASD/HA  and  the 
ASD/RA  direct  the  Director,  Officer  and  Enlisted  Personnel  Management,  to 
develop  and  maintain  a  comprehensive  management  information  system  capa¬ 
ble  of  monitoring  DES  performance  measures  (as  they  apply  to  active  and  Re¬ 
serve  components). 

•  Management  information  system  operator — provide  reports  to  MTF  comman¬ 
ders  and  the  Surgeons  General  on  the  status  of  medical  boards  in  enough  detail 
to  identify  bottlenecks  and  to  highlight  “best  practices.” 

•  Management  information  system  operator — provide  the  Secretaries  of  the 
military  departments  reports  on  the  certification  status  of  primary  participants. 


Process  Intervention 

Process  >4  changes,  by  their  very  nature,  interact  with  changes  in  the  other  categories 
of  interventions.  As  a  result,  several  actions  constituting  the  process  intervention  link 
to  actions  in  other  interventions. 

We  recommend  that  the  OSD  direct  the  military  departments  to  implement  a  proce¬ 
dure  whereby  a  Medical  Evaluation  Board,  upon  deciding  to  forward  a  case  to  the 
PEB,  would  trigger  a  letter  from  the  MTF  commander  to  the  unit  commander.  The 
letter  should  state  the  intent  to  process  the  service  member  through  the  DES.  It 


13 

Output  measures  as.se,ss  immediate  performance  re.sults  of  key  parts  of  the  system  that  contribute  to 
system  outcomes.  They  are  a  mix  of  lagging  and  leading  indicators  of  performance. 

'^The  term  process,  as  used  here,  is  a  particular  method  of  operating  the  DES  involving  a  number  of  steps 
or  operations.  Other  categories  of  interventions  have  focused  on  actions  within  those  specific  steps  or 
operations.  ^ 
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should  by  and  large  be  a  form  letter,  ideally  in  electronic  format,  that  details  the  DBS 
process  and  explains  that  processing  of  the  case  and  replacement  of  the  service 
member  cannot  occur  without  the  commander’s  letter  and  the  LOD  determination. 
The  letter  should  also  refer  the  unit  commander  to  the  proposed  unit  commander 
training  package  located  on  the  proposed  DoD  disability  evaluation  Web  site.  The 
letter  should  also  identify  the  responsible  PEBLO. 

One  practice  we  became  aware  of  during  the  course  of  our  interviews  seems  to  lend 
greater  expertise  to  the  writing  of  narrative  summaries  and  could  have  wider  appli¬ 
cation.  Some  MTFs  designate  and  train  one  physician  (or  several  depending  on  the 
workload)  at  each  facility  to  write  all  narrative  summaries.  Alternatively,  some  MTFs 
employ  retired  physicians  to  carry  out  this  function.  We  recommend  that  the  military 
departments  explore  these  practices  in  greater  depth  for  possible  wider  applicability. 

Several  sources  suggested  that  cases  become  “lost”  while  awaiting  the  compilation  of 
specialty  consults.  The  Air  Force  assigns  responsibility  to  the  initial  contact  physician 
to  ensure  that  the  case  proceeds  through  the  appropriate  consultations.  In  effect,  the 
Air  Force  designates  a  “case  owner.”  We  recommend  that  the  other  military  depart¬ 
ments  assess  the  Air  Force  process  in  terms  of  its  applicability  to  their  own  depart¬ 
ments.  In  addition,  the  departments  should  consider  other  alternatives,  such  as  as¬ 
signing  a  case  to  a  PEBLO  or  patient  administrator  as  soon  as  an  attending  physician 
determines  that  the  service  member  likely  will  require  fitness  evaluation  for  retention 
in  a  duty  status. 

In  order  to  use  the  data  generated  by  a  management  information  system  effectively, 
the  data  must  be  gathered,  evaluated,  and  acted  upon.  With  the  exception  of  the 
Army,  we  found  little  organizational  capability  to  use  information  to  improve  system 
operation.  We  recommend  that  each  military  department  develop  an  organizational 
capability  that  would  enable  it  to  use  data  to  improve  system  operation.  This  capa¬ 
bility  could  reside  at  the  PEB  or,  as  in  the  case  of  the  Army,  in  an  oversight  organiza¬ 
tion.  If  the  recommendation  to  appoint  a  process  owner  in  each  military  department 
is  adopted,  the  capability  should  reside  with  that  individual.  Wherever  the  capability 
resides,  it  should  be  the  basis  for  information  presented  to  senior  officials  responsi¬ 
ble  for  system  oversight.  The  OSD  should  develop  a  similar  capability  to  evaluate 
the  data  across  military  departments  and  components. 

To  summarize  the  Process  intervention: 

•  The  OSD — direct  the  military  departments  to  implement  a  procedure  whereby  a 
Medical  Evaluation  Board  deciding  to  forward  a  case  to  the  PEB  triggers  a  letter 
from  the  MTF  commander  to  the  unit  commander  explaining  the  unit  com¬ 
mander’s  role  in  the  process. 

•  The  military  departments — explore  existing  practices  for  designating  physicians 
with  expertise  in  writing  narrative  summaries  for  wider  applicability. 


proposed  capability  such  as  this  would  help  ensure  consistent  policy  application  within  the  military 
department  and  would  help  facilitate  the  generating  and  monitoring  of  reports,  such  as  comparisons  of 
dispositions  between  officers  and  enlisted  members  among  various  career  fields  and  between  active  and 
Reserve  components. 
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•  The  other  two  military  departments— assess  for  applicability  the  Air  Force  pro¬ 
cess  of  assigning  to  the  initial  contact  physician  responsibility  for  ensuring  a  case 
proceeds  through  the  appropriate  consultations  to  narrative  summary  dictation 
in  a  timely  manner. 

•  Each  military  department— develop  an  organizational  capability  to  use  data  from 
the  management  information  system  to  improve  system  operation. 


Incentives  Intervention 

To  give  military  treatment  facilities  a  greater  incentive  to  assure  that  medical  boards 
from  the  Medical  Evaluation  Board  are  sufficient  before  passing  them  to  the  PEB,  we 
recommend  that  each  PEB  publicly  recognize  each  year’s  best-performing  MTF  by 
presenting  an  award  of  excellence.  This  award  would  be  based,  for  example,  on  the 
percentage  of  medical  boards  deemed  “sufficient  for  adjudication"  by  the  PEB. 

Although  this  recommendation  applies  to  only  one  phase  of  the  disability  evaluation 
process  (based  on  the  issues  identified  during  the  goal  fabric  analysis),  similar  formal 
and  informal  awards  presented  for  top  performance  in  all  phases  of  the  DES  could 
contribute  to  smoother  operation  of  the  system  as  a  whole.  Deployment  of  the  rec¬ 
ommended  management  information  system  would  ensure  that  reliable  data  is 
available  to  serve  as  the  basis  for  selecting  high  performers  for  these  awards. 

To  summarize  the  Incentives  intervention: 

•  Each  PEB — publicly  recognize  the  best-performing  MTF  annually  with  an  award 
of  excellence;  similar  formal  and  informal  awards  throughout  the  system 
contribute  to  smoother  overall  system  operation. 

A  PARADIGM  SHIFT:  FROM  ENSURING  CONSISTENT  POLICY 
APPLICATION  TO  IMPROVING  SYSTEM  PERFORMANCE 

The  focus  of  this  chapter  so  far  has  been  on  issues  regarding  the  consistent 
application  of  disability  policy  and  proposed  actions  to  facilitate  consistent  policy 
application  based  on  our  bottom-up  goal  fabric  analysis.  Although  consistent  appli¬ 
cation  is  an  important  aspect  of  system  performance,  it  does  not  represent  the  whole 
picture. 

All  ten  interventions  listed  earlier  in  this  chapter  are  necessary  to  achieve  more-con¬ 
sistent  application  of  disability  policy.  Two  interventions  in  particular,  however, 
merit  more-extensive  development  than  the  other  interventions  because  of  their 
greater  impact  on  improving  system  performance:  DES  primary  participant  training 
and  implementation  of  a  management  information  system.  In  order  to  develop  the 
most  effective  training  program  and  management  information  system— interven¬ 
tions  intended  to  improve  system  performance  over  time— a  different  approach  or 
methodology  is  needed  in  contrast  to  the  issues-driven,  bottom-up  approach  used  to 
develop  the  interventions  proposed  in  this  chapter. 
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Identifying  interventions  to  improve  DES  performance  ideally  requires  a  “top-down” 
methodology.  Such  an  approach  first  requires  a  commonly  agreed  upon  stated  pur¬ 
pose  for  the  DES,  then  a  desired  set  of  system  outcomes,  and  finally,  for  a  truly  effec¬ 
tive  training  intervention,  a  management  information  system  to  measure  actual  out¬ 
comes  against  desired  outcomes.  In  this  context,  the  differences  between  desired 
and  actual  outcomes  lead  to  the  identification  and  recommendation  of  DES  primary 
participant  training  and  other  interventions  to  eliminate  the  differences.  Figure  4.2 
illustrates  such  a  top-down,  purpose-driven  methodology. 

This  methodology  begins  with  a  clearly  articulated  statement  of  the  purpose  of  the 
DES.  The  desired  system  outcomes  describe  what  successful  system  performance 
would  look  like.  Although  desired  outcomes  are  unlikely  to  be  achieved  quickly  or 
easily  because  they  portray  the  ideal  system  results  that  matter  to  customers — that  is, 
they  “stretch”  the  organization — they  establish  the  basis  for  performance  targets  to 
guide  individual  and  collective  actions,  in  this  case,  the  basis  for  identifying  and  as¬ 
sessing  interventions. 

Unfortunately,  the  foundation  for  developing  these  interventions — that  is,  a  stated 
purpose  for  the  DES  and  identification  of  desired  outcomes,  which  are  needed  to 
employ  this  approach — are  not  available  today  with  respect  to  DES.  That,  in  addition 
to  the  absence  of  a  management  information  system  capable  of  monitoring  system 
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Figure  4.2 — Ideal  Methodology  for  Identifying  Interventions  for  Improved 

System  Performance 
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performance  across  military  departments,  was  the  primary  reason  we  did  not  employ 
the  top-down  methodology  in  this  chapter. 

The  desired  outcomes  suggest  the  sort  of  data  the  management  information  system 
operator  needs  to  gather,  and  the  competencies  that  the  primary  participants  require 
to  perform  their  assignments  effectively.  By  comparing  actual  outcomes  with  desired 
outcomes,  the  management  information  system  facilitates  development  of  training 
and  other  interventions. 

Nevertheless,  to  develop  the  training  intervention  and  specifications  for  a  manage¬ 
ment  information  system  that  can  eventually  assess  the  effectiveness  of  training  and 
other  interventions,  we  demonstrate  later  in  this  report  the  top-down  methodology 
based  on  the  purpose  and  outcomes  suggested  in  Chapter  3.  Chapter  5  describes  the 
training  intervention  in  detail  and  Chapter  6  describes  the  structure  of  a  manage¬ 
ment  information  system.  A  comparison  of  the  two  analytic  methods  (the  bottom- 
up,  issues-driven  goal  fabric  method  and  top-down,  purpose-driven  method)  can  be 
found  in  Appendbc  F. 


Chapter  Five 

DISABILITY  EVALUATION  SYSTEM  TRAINING  ANALYSIS  AND 
RECOMMENDED  CHANGES  IN  TRAINING  TO  IMPROVE 

SYSTEM  PERFORMANCE 


This  chapter  presents  an  extensive  top-down,  purpose-driven  training  needs  analysis 
to  establish  the  recommended  content  and  delivery  method  for  DoD  disability  eval¬ 
uation  training.  The  recommended  training  content  is  designed  to  enable  primary 
participants  to  develop  the  proposed  performance  competencies,  which  are  based 
on  the  set  of  desired  outcomes  and  the  DBS  purpose  statement  we  propose  in 
Chapter  3.  This  chapter  begins  by  introducing  the  methodology  used  and  bodies  of 
DBS-specific  knowledge,  and  then  summarizes  existing  training  activities.  This 
chapter  also  identifies  and  segments  the  target  training  populations  and  suggests 
primary  participant  performance  competencies  based  on  the  stated  purpose  of  the 
DBS  and  the  desired  system  outcomes  as  proposed  in  Chapter  3. 

This  chapter  also  identifies  the  DBS  topics  that  constitute  the  training  content.  The 
DBS  topics  are  grouped  by  primary  participant  population  clusters  (including  unit 
commanders  and  attorneys  who  interact  with  the  system)  that  require  the  necessary 
knowledge  to  produce  desired  on-the-job  results.  Finally,  this  chapter  examines 
other  training  design  considerations  and  closes  with  a  detailed  discussion  of  the  fol¬ 
lowing  recommendations  for  changes  in  training  to  ensure  more-consistent  applica¬ 
tion  of  disability  policy  across  and  within  the  military  departments. 

We  recommend  that  the  Office  of  the  Assistant  Secretary  of  Defense  for  Force 
Management  Policy  champion — that  is,  develop  and  monitor — knowledge-based 
training  in  which  the  content  focuses  on  specific  bodies  of  DBS  knowledge  for  pri¬ 
mary  participant  population  clusters  across  the  military  departments. 

We  further  recommend  delivering  this  knowledge-based  training  through  a  Web  site 
devoted  to  disability  evaluation  training  that  can  be  accessed  by  all  primary  partici¬ 
pants.  This  self-directed  computer-based  distance  training  is  a  basic  course  in  dis¬ 
ability  evaluation,  tailored  to  the  needs  of  each  primary  participant  population  clus¬ 
ter.  More-advanced  classroom  training,  tailored  to  the  needs  of  adjudicators  and 
physicians,  supplements  the  distance  training.  This  training  focuses  on  applying  a 
particular  set  of  DBS  topics  to  develop  the  skills  necessary  to  evaluate  and  adjudicate 
cases  consistently  across  and  within  the  military  departments — a  primary  determi¬ 
nant  of  consistent  application  of  disability  policy. 
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METHODOLOGY 

Developing  a  comprehensive  training  intervention  ideally  begins  with  a  needs  as¬ 
sessment  based  on  system  performance.  This  assessment  first  requires  an  agreed- 
upon  statement  of  organizational  intent.  In  this  particular  case,  it  requires  a  purpose 
statement  for  the  DES,  together  with  a  set  of  desired  system  outcomes  and  a  man¬ 
agement  information  system  capable  of  monitoring  and  assessing  how  well  the  sys¬ 
tem  accomplishes  its  overall  purpose.  Unfortunately,  as  noted  earlier  in  this  report, 
these  prerequisites  do  not  currently  exist.  Therefore,  given  our  exposure  to  the  issues 
presented  by  primary  participants,  our  observation  of  current  military  department 
training  events,  and  our  study  of  policy  documents,  we  formulated  a  DES  purpose 
statement  and  a  set  of  desired  system  outcomes,  which  are  presented  in  Chapter  3, 
that  serve  as  the  basis  for  the  training  recommendations  in  this  chapter. 

We  propose  a  specific  purpose  and  set  of  desired  outcomes  in  order  to  present  a 
methodology  for  developing  the  training  intervention  and  a  set  of  metrics  for  use  in  a 
management  information  system.  We  formulated  suggested  performance  compe¬ 
tencies  for  each  of  the  primary  participant  populations  across  the  military  depart¬ 
ments  to  better  focus  our  training  needs  assessment  and  recommendations. 

However,  based  on  our  study,  we  believe  the  OSD  should  determine  a  DES  statement 
of  purpose  and  a  set  of  desired  outcomes,  and  then  reapply  the  methodology  to  the 
purpose  and  desired  outcomes  that  its  leaders  produce.  Decisions  on  the  purpose 
statement,  the  desired  outcomes,  and  primary  participant  competencies  inform  the 
specific  training  content  and  training  packages  tailored  to  the  needs  of  primary  par¬ 
ticipant  population  clusters. 

As  noted  in  Chapter  4,  we  recommend  that  as  a  first  step,  the  ASD/FMP  direct  the 
Director  of  Officer  and  Enlisted  Personnel  Management  to  consult  with  a  small  group 
of  experienced  DES  experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General,  to  produce  recommendations  upon  which  the  ASD/FMP,  in 
coordination  with  the  ASD/HA  and  ASD/RA,  can  decide  upon  a  statement  of  the 
purpose  and  DoD’s  desired  outcomes  of  the  DES.  This  statement  of  purpose  and  the 
desired  outcomes,  in  turn,  shape  the  performance  competencies  required  by  the 
primary  participant  populations  and  the  specific  training  content,  which  ideally, 
emerges  from  additional  group  consultations. 

This  report  presents  an  integrated  methodology  to  serve  as  both  a  starting  point  and 
a  guide  for  final  determination  of  training  content  and  delivery.  This  chapter,  in  par¬ 
ticular,  demonstrates  this  methodology  using  the  DES  purpose  and  desired  system 
outcomes  proposed  in  Chapter  3. 

BODIES  OF  DES-SPECIFIC  KNOWLEDGE 

Executing  DoD  disability  policy  through  the  three  military  departments’  Disability 
Evaluation  Systems  requires  primary  participants  to  interpret,  at  an  operational  level. 
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hundreds  of  specific  requirements  derived  from  various  statutes  ^  and  policy  docu¬ 
ments.  Assignment  to  positions  within  any  of  the  first  three  phases  of  the  DES — 
medical  evaluation;  physical  disability  evaluation,  including  both  the  Informal  PEB 
and  the  Formal  PEB;  and  appellate  review  at  a  level  beyond  the  PEB^ — requires 
specific  bodies  of  knowledge  and  skills  to  interpret  and  apply  disability  policy- 
knowledge  and  skills  generally  not  acquired  in  other  assignments. 

As  a  result,  every  person  assigned  to  a  position  or  designated  for  duty  in  any  of  these 
three  phases  of  the  DES  requires  a  new  body  of  knowledge  based  on  a  set  of  DES  top¬ 
ics  and  new  skills  to  produce  the  desired  on-the-job  results.  Active  and  Reserve  com¬ 
ponent  unit  commanders  who  interact  with  the  system  and  attorneys  who  represent 
service  members  during  appeals  also  require  a  new  body  of  knowledge  based  on  the 
DES  topics. 

Producing  the  desired  on-the-job  results  also  requires  other  abilities,  characteristics, 
traits,  and  behaviors  that  individuals  may  acquire  in  other  assignments,  such  as  the 
ability  to  apply  quality  principles,  operate  automated  systems,  write  clearly,  collect 
and  organize  information  according  to  a  stated  standard,  and  the  ability  to  project  a 
cooperative  team  spirit,  professional  image,  and  positive  attitude  within  an  office 
environment.  This  chapter  focuses  on  the  need  for  DES  bodies  of  knowledge  and 
skills. 

We  accept  the  premise  that  individuals  who  possess  specific  bodies  of  knowledge 
and  skills  related  to  their  jobs  in  the  DES  are  more  likely  to  consistently  apply  disabil¬ 
ity  policy.  Therefore,  we  believe  training  that  enables  participants  to  produce  the  de¬ 
sired  on-the-job  results  is  a  critical  management  intervention  for  ensuring  consistent 
policy  application. 

Current  Disability  Evaluation  Training  Within  the  Department  of  Defense 

The  OSD  currently  provides  no  disability  evaluation  training;  however,  it  does  spon¬ 
sor  quarterly  meetings  for  the  senior  leaders  of  the  military  departments’  PEBs  and 
the  medical  communities  that  compose  the  DES,  the  legal  communities  that  interact 
with  the  system,  and  other  interested  parties.^ 

Currently,  individuals  assigned  to  positions  within  the  first  three  phases  of  the  DES, 
in  addition  to  unit  commanders  who  interact  with  the  system,  acquire  knowledge  on 
new  DES  topics  and  gain  new  skills  primarily  through  ad  hoc  on-the-job  training.  In 
virtually  all  cases  across  the  military  departments,  newly  assigned  individuals  start 
performing  their  jobs  immediately  and  learn  new  DES  topics  and  skills  bit  by  bit  on  a 
case-by-case  basis.  As  described  by  primary  participants  during  our  interviews  with 


^Those  statutes  include  Section  104  of  Title  26,  U.S.C.,  Chapter  61,  U.S.C.,  Sections  801-940  of  Title  10, 
U.S.C.:  Sections  3502,  5532,  6303,  and  8332  of  Title  5,  U.S.C.;  Sections  206  and  502  of  Tide  37,  U.S.C.;  and 
Sections  101  and  302  of  Title  38,  U.S.C. 

^The  DES  is  composed  of  four  phases.  The  fourth  phase  is  final  disposition. 

^See  Footnote  8  in  Chapter  4. 
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them,  management  seems  to  accept  mistakes  made  in  the  course  of  learning  and  any 
inconsistencies  in  policy  application  that  may  result  as  a  cost  of  doing  business. 

Each  of  the  military  departments’  PEBs  provides  some  classroom  training  targeted  to 
its  PEBLO  population  (ranging  from  one  to  two  weeks  on  an  annual  or  quarterly 
schedule),  to  which  they  invite  other  interested  primary  participants,  including  at¬ 
torneys.  The  Army  Physical  Disability  Agency  is  the  only  military  department  that 
develops  and  once  a  year  or  so  conducts  comprehensive  classroom  training. 
Although  this  training  is  targeted  to  the  needs  of  Army  adjudicators,  the  agency  in¬ 
vites  other  primary  participants  and  offers  limited  seats  to  adjudicators  from  the 
other  military  departments.  The  Air  Force  Disability  Division  convenes  quarterly 
meetings  for  all  Air  Force  Informal  PEB  and  Formal  PEB  members,  which  are  also  at¬ 
tended  by  an  appellate  review  board  member.  Meeting  participants  discuss  cases, 
problem  areas,  and  the  process  by  which  they  make  decisions.  The  Army  established 
a  list  serve  for  PEBLOs,  and  the  Department  of  the  Navy  provides  a  PEBLO  training 
video. 

PEB  members,  approving  authorities,  and  administrative  action  officers  in  all  three 
military  departments  take  the  initiative  to  share  information  with  their  respective 
department’s  medical  community — the  physicians  who  write  narrative  summaries 
and  who  serve  as  Medical  Evaluation  Board  members  and  approving  authorities — in 
person.  PEB  members,  approving  authorities,  and  administrative  action  officers  visit 
a  few  of  these  physicians  each  year  to  provide  training  to  help  them  prepare  medical 
boards  that  are  sufficiently  detailed,  timely,  and  complete  for  PEB  adjudication. 

At  some  military  treatment  facilities,  patient  administrators  or  the  Medical 
Evaluation  Board  approving  authorities  advise  physicians  on  how  to  dictate  suffi¬ 
ciently  detailed  and  complete  narrative  summaries  and  specialty  consults.  The  vari¬ 
ous  military  treatment  facilities  handle  physician  training  differently,  and  may  range 
from  personal  mentoring  to  brown  bag  lunches  to  locally  developed  guidelines.  For 
example,  the  President  of  the  Department  of  the  Navy  PEB  and  the  PEB  administra¬ 
tive  action  officers  present  a  half-day  briefing  to  senior  enlisted  members  during  the 
Navy  s  course  on  patient  affairs.  Additional  ideas  for  improving  primary  participant 
training  and  system  performance  are  in  various  stages  of  development  in  all  three 
military  departments. 

The  PEB  approving  authorities^  in  all  three  military  departments  champion  the  DES. 
They  continuously  urge  collaboration  among  primary  participants  and  admonish 
their  subordinates  to  “communicate,  communicate,  communicate.”  All  of  the  mili¬ 
tary  departments  have  also  established  some  sort  of  information-sharing  presence 
on  the  World  Wide  Web. 

Clearly,  the  military  departments  collectively  employ  a  wide  range  of  training  meth¬ 
ods  to  enable  primary  participants  to  produce  desired  on-the-job  results;  on-the-job 
training;  mentoring  and  coaching;  self-directed  learning,  including  videos  and  early- 


^Those  authorities  include  the  Deputy  Commander,  Army  Physical  Disability  Agency:  President,  Physical 
Evaluation  Board  (Department  of  the  Navy);  and  Chief,  Air  Force  Physical  Disability  Division. 
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Stage  computer-based  distance  training;  seminars  and  workshops;  and  formal  class¬ 
room  training,  including  case  studies.  Lacking  clear  direction  from  the  OSD,  how¬ 
ever,  each  military  department  conducts  training  to  meet  its  own  recognized  needs 
based  on  its  interpretation  of  policy  with  little  regard  for  training  to  enhance  the 
consistency  of  disability  policy  across  the  military  departments. 

Because  the  objective  of  this  study  was  to  identify  and  recommend  changes  to  train¬ 
ing  provided  to  the  primary  participants  of  the  DES,  we  began  with  the  current  train¬ 
ing  practices  and  then  focused  on  recommended  improvements.  The  first  step  in  this 
process  entails  identifying  the  target  population. 

Target  Population 

Training  is  effective  only  to  the  degree  that  it  produces  the  desired  behavior^  in  the 
population  trained.  We  identified  12  primary  participant  populations  who  require 
specific  bodies  of  knowledge  and  skills  to  execute  disability  policy  throughout  the 
military  departments: 

1.  Physical  Evaluation  Board  liaison  officers  (PEBLOs),  and  disability  evaluation 
counselors  (in  the  Department  of  the  Navy) 

2.  Patient  administrators  (who  assist  PEBLOs  at  military  treatment  facilities) 

3.  Physicians  at  MTFs  (who  write  narrative  summaries  and  specialty  consults  for 
medical  boards) 

4.  Medical  Evaluation  Board  members  at  MTFs 

5.  Medical  Evaluation  Board  approving  authorities  at  MTFs 

6.  Physical  Evaluation  Board  administrative  action  officers  (who  quality  check  and 
process  medical  boards) 

7.  Physical  Evaluation  Board  members 

8.  Physical  Evaluation  Board  approving  authorities® 

9.  Appellate  review  board  members  beyond  tbe  Formal  PEB 

10.  Active  component  unit  commanders  (who  interact  with  the  DES) 

1 1 .  Reserve  component  unit  commanders  (who  interact  with  the  DES) 

12.  Attorneys  (who  represent  and  advise  service  members). 


^Desired  behavior  has  been  defined  as  an  action  or  response  to  a  situation  in  which  a  performer  uses 
certain  knowledge  and  skills  to  bring  about  a  desired  result  (Shapiro,  1995). 

®These  approving  authorities  include  the  Deputy  Commander,  Army  Physical  Disability  Agency; 
President,  Physical  Evaluation  Board  (Department  of  the  Navy);  and  Chief,  TUr  Force  Physical  Disability 
Division. 
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Characteristics  of  Target  Population.  Predominant  characteristics  of  the  target 
training  population  impact  training  design.  The  distinguishing  characteristics  of  the 
DES  target  training  population  include: 

•  Frequent  turnover  among  many  primary  participants 

•  Dispersion  of  primary  participants  across  the  United  States 

•  Wide  variation  of  subject  matter  expertise  among  primary  participant  popula¬ 
tions 

•  Wide  variation  in  the  amount  of  required  disability  evaluation  detailed  knowl¬ 
edge  among  different  primary  participant  populations 

•  A  high  level  of  computer  literacy  and  comfort  with  using  computer  systems. 

Turnover  Among  Primary  Participants.  Most  of  the  primary  participant  populations 
consist  of  military  members  and  are,  therefore,  subject  to  frequent  reassignment  and 
relocation,  often  every  two  to  three  years.  Whereas  Departments  of  the  Army  and 
Navy  physicians  serve  as  Medical  Evaluation  Board  members  at  virtually  any  time 
required.  Department  of  the  Air  Force  physicians  generally  serve  on  Medical 
Evaluation  Boards  on  an  ad  hoc  basis,  rotating  in  and  out  of  board  duty  during  a 
normal  assignment  to  an  MTF. 

Air  Force  PEBLOs  generally  rotate  in  and  out  of  PEBLO  duty  during  a  two-  to  three- 
year  tour  to  an  MTF.  Within  the  Army  DES,  some  PEB  physician  members  and  most 
PEBLOs  (assigned  to  CONUS)  are  Department  of  the  Army  civilians  and  therefore, 
turn  over  infrequently.  Similarly,  nine  out  of  ten  Air  Force  PEB  action  officers  and 
some  Department  of  the  Navy  disability  evaluation  counselors  are  military 
department  civilians  and  therefore  turn  over  infrequently.  The  military  departments 
reported  on  approximate  tour  lengths  among  the  primary  participant  populations, 
which  are  shown  in  Table  5.1. 

Geographic  Dispersion.  Physicians  are  the  most  widely  dispersed  population  be¬ 
cause  virtually  any  physician  in  an  MTF  may  write  a  narrative  summary.  PEBLOs, 
patient  administrators,  and  physicians  who  serve  on  Medical  Evaluation  Boards  are 
also  widely  dispersed  wherever  military  treatment  facilities  exist.  As  an  example. 
Figure  5.1  illustrates  dispersion  of  Department  of  the  Navy  PEBLOs  and  disability 
evaluation  counselors. 

PEB  members,  approving  authorities,  and  administrative  action  officers  are  located 
at  Randolph  Air  Force  Base  and  Lackland  Air  Force  Base,  Texas;  Fort  Lewis, 
Washington;  Fort  Sam  Houston,  Texas;  San  Diego,  California;  Bethesda,  Maryland; 
and  Washington,  D.C.  Likewise,  attorneys  who  represent  service  members  during 
appeals  are  currently  dispersed  over  the  same  locations  as  the  formal  PEBs.  Active 
and  Reserve  component  unit  commanders  who  interact  with  the  DES  from  poten¬ 
tially  every  military  installation  and  deployment  are  the  most  dispersed  populations. 
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Table  5.1 

Approximate  Tour  Lengths  of  Primary  Participants 


Primary  Participants 

Department  of  the 
Army 

Department  of  the 
Navy 

Department  of  the 

Air  Force 

Physicians  Who  Write 
Narrative  Summaries 

3-4  years 

3-4  years 

Ad  hoc  duty  during  3- 
to  4-year  tour 

PEBLOs 

3  years  (military, 
OCONUS) 

2-20  years  (civilian, 
CONUS) 

3  years 

Commonly  rotate 
through  PEBLO  duty 
during  2-  to  3-year 
tour  at  an  MTF 

Disability  Evaluation 
Counselors 

N/A 

1-3  years  (military) 
3-15  years  (civilian) 

N/A 

Patient  Administrators 

2-3  years 

2-3  years 

2  years 

Medical  Evaluation  Board 
Members 

2-3  years 

2-3  years 

Rotate  physicians 
through  Medical 
Evaluation  Board 
duty  during  normal 
tour 

Medical  Evaluation  Board 

2  years 

2-3  years 

2-3  years 

Approving  Authorities 

PEB  Administrative  Action 

Civilian 

1-3  years 

Civilian 

Officers 

Physical  Evaluation  Board 

1-2  years 

1-3  years 

3  years 

Members 

Physical  Evaluation  Board 

1-2  years 

6  months 

1-2  years 

Approving  Authorities 

Post-PEB  Appellate 

Review  Board  Members 

3  years 

3  years 

Ad  hoc  duty  during 

3 -year  tour 

Attorneys 

1-2  years 

1-3  years 

3-4  years 

Unit  Commanders 

12-18  months 

2-3  years 

— 

Subject  Matter  Expertise.  All  primary  participants  bring  some  level  of  expertise  and 
specialty  knowledge  of  certain  subject  matter  to  their  positions  within  the  DES. 
Physicians — generally  interns  and  first-year  residents — who  write  narrative  sum¬ 
maries,  as  well  as  those  who  write  specialty  consults  and  those  who  serve  on  Medical 
Evaluation  Boards  and  PEBs,  bring  extensive  health  care  and  medical  expertise  to  the 
DES.  Likewise,  attorneys  who  advise  service  members,  normally  as  their  first  assign¬ 
ment  after  completing  law  school,  bring  legal  expertise  to  the  DES. 

Officers  assigned  as  PEB  approving  authorities  (0-6s),  nonmedical  members  of  the 
PEB  (generally  0-5s),  and  those  who  serve  ad  hoc  on  appellate  review  boards,  all  of 
whom  are  typically  serving  their  last  tour  before  retirement,  bring  their  depth  of  ex¬ 
pertise  as  senior  military  leaders  from  a  variety  of  subject  matter  backgrounds. 

Long-serving  PEBLOs,  patient  administrators,  and  PEB  administrative  action  officers 
bring  expertise  in  their  occupational  specialties  based  on  their  years  of  experience. 
Department  of  the  Navy  military  disability  evaluation  counselors  and  Air  Force 
PEBLOs  probably  have  the  greatest  variation  in  terms  of  the  levels  and  range  of  sub¬ 
ject  matter  expertise.  Lastly,  unit  commanders,  typically  0-3s,  bring  unique  com¬ 
mand  and  varied  subject  matter  expertise. 
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Figure  5.1 — Example  of  Geographic  Dispersion:  Department  of  the  Navy  PEBLOs  and 
Disability  Evaluation  Counselors  with  Collateral  Duty 


Noticeably,  this  vast  range  of  subject  matter  expertise  also  reflects  great  diversity  in 
educational  backgrounds  and  learning  experiences.  But  more  significantly,  few,  if 
any,  primary  participants  typically  bring  DES-specific  subject  matter  expertise  to 
their  positions  in  the  DBS. 

Variation  in  Level  of  Required  Disability  Evaluation  Expertise.  Although  different 
primary  participant  populations  require  identical  levels  of  understanding  of  many 
DES  topics,  such  as  the  DES  purpose  and  desired  system  outcomes,  some  require 
vastly  different  levels  of  detailed  understanding  in  order  to  produce  the  desired  re¬ 
sults  in  their  various  jobs.  For  example,  physicians  who  serve  on  PEBs  require  an  in- 
depth  understanding  of  how  to  apply  VASRD  codes  to  the  wide  range  of  medical  di- 
agnoses  presented  in  medical  boards  and  how  to  rate  diagnoses  by  analogy.^ 
Likewise,  PEBLOs  require  sufficient  understanding  of  how  to  apply  VASRD  codes  and 
rating  by  analogy  to  explain  to  service  members  what  it  would  take  to  warrant  a  dif¬ 
ferent  VASRD  or  analogy  code.  By  contrast,  patient  administrators,  PEB  administra¬ 
tive  action  officers,  and  unit  commanders,  for  example,  only  require  a  limited  un¬ 
derstanding  of  VASRD  and  analogous  codes. 


y 

When  a  medical  condition  is  not  listed  in  the  VASRD,  physicians  who  serve  on  the  PEB  may  use  a  specific 
procedure  to  rate  it  under  a  closely  related  disease  or  injury  in  which  not  only  the  affected  functions,  but 
also  the  anatomical  localization  and  symptomatology,  are  closely  analogous. 
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Computer  Literacy.  Today’s  high-tech  military  relies  heavily  on  computers  and,  as  a 
result,  produces  a  computer-literate  workforce,  both  military  and  civilian.  Individ¬ 
uals  assigned  to  the  DES  generally  possess  computer  skills. 

These  target  training  population  characteristics  appear  again  with  the  design  consid¬ 
erations  presented  later  in  this  chapter.  The  next  section  suggests  performance  com¬ 
petencies  for  primary  participant  populations  across  the  military  departments. 


Competencies  Derived  from  Desired  Outcomes 

Ideally,  a  training  intervention  is  closely  tied  to  the  organization’s  job  analysis  and 
design,  assignment,  and  individual  performance  management  practices.  Further,  the 
intervention  requires  assessing  the  existing  performance  competencies  including 
bodies  of  knowledge,  skills,  abilities,  characteristics,  traits,  and  behaviors  exhibited 
by  individuals  assigned  to  the  positions  and  then  comparing  their  individual  and 
group  performance  to  a  set  of  desired  results  and  behaviors. 

Job  analysis,®  the  preferred  technique  to  assess  competencies,  is  time-consuming 
and  beyond  the  scope  of  this  project.  It  should  be  undertaken  only  after  the  OSD  it¬ 
self  establishes  a  stated  DES  purpose  and  set  of  desired  outcomes,  which  would  serve 
as  guidelines  for  the  analysis.  Ideally,  developing  training  that  produces  desired  be¬ 
haviors  and  desired  on-the-job  results  from  the  population  trained  requires  the  ca¬ 
pability  to  monitor  system  performance  and  system  results  and  compare  them  to  a 
set  of  stated  desired  results.  Chapter  6  describes  a  management  information  system 
that  could  serve  this  role  in  the  future. 

The  DoD  desired  system  outcomes,  which  we  propose  in  Chapter  3  and  present 
again  here  as  follows,  shape  the  competencies  required  for  primary  participants  to 
produce  the  desired  on-the-job  results: 

1.  Service  members  having  a  similar  condition  and  similar  office,  grade,  rank,  or 
rating  receive  similar  fitness  decisions  within  the  military  department. 

2.  Service  members  found  unfit  receive  similar  disability  ratings  for  similar 
conditions  across  and  within  the  military  departments. 

3.  Service  members  freely  and  appropriately  exercise  their  rights  to  administrative 
due  process. 

4.  Service  members  return  to  duty,  separate  for  disability,  or  retire  for  disability  in  a 
timely  manner. 


®/ob  analysis,  also  known  as  Job-task  analysis,  is  the  examination  of  the  parts  of  a  job.  In  this  case,  it  is  an 
analysis  of  the  parts  of  the  jobs  of  the  primary  participant  populations:  physicians  who  write  narrative 
summaries;  PEBLOs  and  disability  evaluation  counselors;  patient  administrators;  Medical  Evaluation 
Board  members  and  approving  authorities;  PEB  administrative  action  officers,  members,  and  approving 
authorities;  appellate  review  board  members  beyond  the  PEB;  and  unit  commanders  (regarding 
interaction  with  the  DES).  Job  analysis  entails  understanding  the  desired  job  results,  the  sequence  of  the 
parts  of  the  job,  the  frequency  of  performance  of  the  parts,  their  criticality  to  successful  performance,  and 
accompanying  bodies  of  knowledge. 
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5.  Primary  participants  perform  their  duties  as  efficiently  as  possible  so  that,  col¬ 
lectively,  they  return  service  members  to  duty,  or  separate  or  retire  service 
members  for  disability  in  a  fair,  consistent,  and  timely  manner,^ 

Given  the  existing  job  designs  for  primary  participant  populations,  we  translated 
these  statements  of  desired  system  outcomes  into  the  following  statements  that  de¬ 
scribe  the  major  activities  that  the  primary  participant  populations  must  be  able  to 
perform  to  achieve  the  desired  system  outcomes: 

Members  of  Medical  Evaluation  Boards 

•  Apply  disciplined  medical  retention  standards  uniformly  such  that  members 
having  a  similar  condition  and  similar  office,  grade,  rank,  or  rating  receive  similar 
medical  retention  decisions 

•  Apply  the  correct  rules  in  a  disciplined  manner  and  document  the  substantial 
evidence  that  supports  the  decision  to  refer  the  service  member  to  a  PEB. 

Members  of  Physical  Evaluation  Boards  and  Appellate  Review  Boards 

•  Uniformly  apply  fitness  standards  such  that  members  having  a  similar  condition 
and  a  similar  office,  grade,  rank,  or  rating  receive  similar  fitness  decisions  within 
the  military  department 

•  Uniformly  rate  service  members  found  unfit,  such  that  members  having  a  similar 
condition  and  a  similar  office,  grade,  rank,  or  rating,  receive  similar  disability 
ratings  across  and  within  the  military  departments 

•  Apply  the  correct  rules  in  a  disciplined  manner  and  document  the  substantial 
evidence  that  supports  all  decisions  (fitness,  rating,  and  disposition). 

PEBLOs  and  attorneys 

•  Provide  complete  and  accurate  information  so  that  service  members  have  suffi¬ 
cient  understanding  of  the  process  and  their  rights  so  that  they  may  exercise 
their  rights  to  due  process  under  the  law. 

All  primary  participants 

•  Perform  their  duties  as  effectively  as  possible  so  that,  collectively,  they  return 
service  members  to  duty,  separate  service  members  for  disability,  or  retire  them 
for  disability  in  a  timely  manner. 

These  statements  shaped  the  formulation  of  the  following  competencies  for  ten  of 
the  primary  participant  populations.  The  following  suggested  competencies,  in  turn, 
help  to  better  focus  the  training  intervention  recommendations: 


®The  fifth  proposed  desired  outcome  is  so  pervasive  that  it  does  not  link  directly  to  individual  primarv- 
participant  population  competencies  or  to  specific  training  content;  nonetheless,  as  an  intended  result  of 
Iterating  the  system  to  achieve  its  stated  purpose,  it  influences  training  needs  assessment  and  design.  In 
this  particular  case,  the  collective  training  recommendations  contribute  to  achieving  this  desired  system 
outcome.  ' 
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Physicians  who  write  narrative  summaries 

•  Able  to  determine  the  appropriate  diagnosis 

•  Able  to  determine  if  service  member’s  condition  calls  into  question  his  or  her 
ability  to  meet  medical  retention  standards 

•  Able  to  synthesize  a  service  member’s  medical  evidence  from  all  appropriate 
consultations  into  a  single  narrative  summary  that  contains  sufficient  informa¬ 
tion  in  the  appropriate  format  for  a  PEB  to  adjudicate  the  case. 

PEBLOs  and  Disability  Evaluation  Counselors 

•  Able  to  accurately  advise  service  members  on  the  process  and  of  their  rights, 
benefits  and  entitlements,  and  wbat  to  expect  as  the  service  member’s  medical 
board  progresses  through  the  DES 

•  Able  to  gather  and  process  patient  information  to  assemble  medical  boards  (case 
files)  that  contain  sufficient  information  in  the  appropriate  format  for  a  PEB  to 
adjudicate  the  case. 

Patient  Administrators 

•  Able  to  assist  Medical  Evaluation  Boards  and  PEBLOs  in  gathering  and  process¬ 
ing  patient  information  to  assemble  medical  boards  that  contain  sufficient  in¬ 
formation  in  the  appropriate  format  for  a  PEB  to  adjudicate  the  case. 

Medical  Evaluation  Board  Members 

•  Able  to  determine  whether  the  medical  board  includes  appropriate  specialty 
consultations  with  sufficient  information 

•  Able  to  determine  the  duty  limitations  associated  with  the  diagnosis 

•  Able  to  determine  whether  the  service  member  meets  the  military  department’s 
medical  retention  standards  for  continued  military  duty. 

Medical  Evaluation  Board  Approving  Authorities 

•  Able  to  identify  complete  and  accurate  medical  boards 

PEB  Administrative  Action  Officers 

•  Able  to  ensure  contents  of  medical  boards  received  by  the  PEB  are  complete  and 
accurate  for  adjudication 

•  Able  to  obtain  missing  information,  monitor,  and  move  medical  boards  through 
the  system,  and  exchange  information  with  PEBLOs. 

Physical  Evaluation  Board  Members 

•  Able  to  apply  disciplined  military  department  fitness  standards  in  a  uniform 
manner 
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•  Able  to  apply  other  rules  uniformly  such  that  members  having  a  similar  condi¬ 
tion  and  a  similar  office,  grade,  rank,  or  rating  receive  similar  disability  ratings 
across  and  within  the  military  departments 

•  Able  to  document  the  substantial  evidence  that  supports  all  PEB  decisions. 
Physical  Evaluation  Board  Approving  Authorities 

•  Able  to  identify  correct  and  consistent  application  of  military  department  fitness 
standards  such  that  members  having  a  similar  condition  and  a  similar  office, 
grade,  rank,  or  rating,  receive  similar  fitness  decisions  within  the  military 
department 

•  Able  to  identify  correct  and  consistent  application  of  other  rules  such  that  mem¬ 
bers  having  a  similar  condition  receive  similar  disability  ratings  across  and 
within  the  military  departments 

•  Able  to  identify  sufficient  documentation  of  the  substantial  evidence  that  sup¬ 
ports  all  PEB  decisions. 

Post-PEB  Appellate  Review  Board  Members 

•  Able  to  apply  disciplined  military  department  fitness  standards  in  a  uniform 
manner 

•  Able  to  apply  other  rules  uniformly  such  that  members  having  a  similar  condi¬ 
tion  receive  similar  disability  ratings  across  and  within  the  military  departments 

•  Able  to  document  the  substantial  evidence  that  supports  all  decisions. 

Unit  Commanders 

•  Able  to  provide  written  evidence  with  sufficient  detail  for  PEB  consideration  that 
documents  his  or  her  judgment  of  how  a  service  member’s  medical  condition 
impacts  the  member’s  ability  to  perform  the  duties  of  the  member’s  office,  grade, 
rank,  or  rating  including  his  or  her  ability  to  deploy,  and  pending  adverse 
actions. 

We  consider  attorneys  who  advise  and  represent  service  members  and  unit  com¬ 
manders  (both  active  and  Reserve  component)  who  interact  with  the  DES  as  popula¬ 
tions  external  to  the  system.  Primary  participants  raised  no  issues  regarding  Reserve 
component  commanders  or  due  process;  therefore,  we  did  not  focus  on  articulating 
competencies  for  Reserve  commanders  and  attorneys.  Nevertheless,  primary  partic¬ 
ipants  across  the  military  departments  identified  active  component  unit  comman¬ 
ders  as  a  major  source  of  delay  in  processing  cases;  therefore,  we  formulated  a  unit 
commander  competency. 

Establishing  common  competencies  for  primary  participant  populations  across  mili¬ 
tary  departments  is  a  critical  factor  in  developing  a  DoD-ivide  training  intervention 
that  leads  to  more-consistent  application  of  DoD  disability  policy;  as  stated  earlier  in 
this  chapter,  job  analysis  is  the  preferred  technique.  These  suggested  competencies 
point  to  a  training  emphasis  on  DES  topics. 
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Training  Content:  DES  Topics 

The  DoD  desired  system  outcomes  not  only  shape  the  performance  competencies 
for  individuals  assigned  to  the  DES  but  also  point  to  a  DoD  emphasis  on  DES  topics 
and  skills  in  applying  knowledge  of  those  topics,  across  the  military  departments. 
Nearly  all  of  the  DES  topics  suggested  in  this  report  emanate  directly  from  the  gov¬ 
erning  statutes  and  OSD  policy  documents.  In  the  few  instances  in  which  they  do 
not,  they  flow  directly  from  other  management  interventions  recommended  in  this 
report— interventions  that  logically  precede  implementation  of  the  recommended 
training  interventions. 

This  OSD-sponsored  study  focuses  on  recommending  changes  to  training  in  order  to 
ensure  more  consistent  application  of  disability  policy.  The  OSD  focus  on  consistent 
policy  application  suggests  that  DES  topics  and  the  associated  skills  required  to  apply 
knowledge  of  those  topics  are  the  most  relevant  aspects  of  a  DoD  training  interven¬ 
tion. 

As  a  result,  we  compiled  a  comprehensive  list,  which  appears  in  Appendix  G,  of  DES 
topics  from  policy  documents  and  current  military  department  disability-training 
syllabi.  We  associated  each  suggested  topic  with  the  primary  participant  populations 
who  require  knowledge  of  that  topic  to  produce  the  desired  on-the-job  results,  rec¬ 
ognizing  that  different  populations  may  apply  the  same  knowledge  differently  in 
their  respective  jobs.  The  list  in  Appendix  G  highlights  the  scope  of  suggested  DES 
subjects  required  by  each  primary  participant  population.  Figure  5.2  shows  the  per¬ 
centage  of  DES  topics  in  which  each  primary  participant  population  must  possess 
some  expertise.  Although  the  figure  depicts  the  quantity  of  DES  topics,  it  does  not 
provide  an  indication  of  the  complexity  of  those  particular  topics  or  how  deeply  the 
participant  population  needs  to  know  any  given  topic  within  its  associated  body  of 
knowledge. 

The  proposed  competencies  introduced  earlier  in  this  chapter,  together  with  an 
analysis  of  DES  topics,  indicates  that  the  primary  participant  populations  listed 
within  the  five  clusters  shown  in  Table  5.2  require  different  levels  of  knowledge  of 
essentially  the  same  DES  topics.  This  analysis  suggests  designing  the  training 
content — that  is,  the  DES  topics — in  five  distinct  training  packages,  one  per 
population  cluster,  as  described  later  in  this  chapter. 

It  is  important  to  restate  that  different  primary  participants  apply  the  same  knowl¬ 
edge  in  different  ways  to  produce  the  desired  results  in  their  respective  jobs.  In  other 
words,  different  primary  participant  populations  require  different  skills  to  apply  dif¬ 
ferent  levels  of  knowledge  of  DES  topics  to  their  respective  jobs.  For  example, 

•  Physicians  who  write  narrative  summaries  apply  their  knowledge  of  “sufficient 
narrative  summaries”  to  dictating  the  kind  of  detailed  information  and  format¬ 
ting  needed  by  the  PEB  to  adjudicate  a  case. 

•  Patient  administrators  apply  their  knowledge  of  “sufficient  narrative  summaries" 
to  formatting  the  data  dictated  by  the  physician. 
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Figure  5,2  Percentage  of  Disability  Evaluation  System-Specific  Topics  in  Which  Primary 

Participants  Must  Exhibit  Expertise 


Table  5.2 

Primary  Participant  Population  Clusters 


Population  Cluster  1  PEBLOs  and  disability  evaluation  counselors 

Patient  administrators 

PEB  administrative  action  officers 


Population  Cluster  2 


Population  Cluster  3 


Population  Cluster  4 


Physicians  who  write  narrative  summaries 
Medical  Evaluation  Board  members 
Medical  Evaluation  Board  approving  authorities 

PEB  members 

PEB  approving  authorities^ 

Post-PEB  Appellate  Review  Board  members 
Attorneys  who  represent  service  members 

Active  component  unit  commanders 


Population  Cluster  5 _ Reserve  component  commanders 

These  authorities  are  the  Deputy  Commander,  Army  Physical  Disability  Agency;  President,  Physical 
Evaluation  Board  (Department  of  the  Navy) ;  and  Chief,  Air  Force  Physical  Disability  Division. 
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♦  PEBLOs  and  disability  evaluation  counselors,  patient  administrators,  and  PEB 
administrative  action  officers  apply  their  knowledge  of  “sufficient  narrative 
summaries”  to  perform  a  quality  check  on  the  medical  board.  They  compare  the 
physician’s  narrative  description  to  the  individual  specialty  consultations  and 
specialty  measurements  required  for  the  PEB  to  adjudicate  the  case;  they  ensure 
the  referring  physician’s  specialty  matches  the  type  of  diagnosis;  and  they  moni¬ 
tor  the  date  signed. 

♦  The  PEB  administrative  action  officers  apply  their  knowledge  of  “sufficient  nar¬ 
rative  summaries”  to  turn  back  insufficient  medical  boards  for  correction  and 
forward  sufficient  medical  boards  to  the  PEB  for  adjudication. 

♦  PEB  members  apply  their  knowledge  of  “sufficient  narrative  summaries”  to  turn 
back  insufficient  medical  boards  and  adjudicate  sufficient  ones. 

In  summary,  the  suggested  purpose  statement  for  DES  and  the  desired  system  out¬ 
comes  presented  in  this  report,  and  the  issues  highlighted  by  the  primary  partici¬ 
pants,  shaped  the  suggested  competencies  for  the  primary  participant  populations. 
The  proposed  purpose,  desired  outcomes,  participant  issues,  and  proposed  compe¬ 
tencies,  in  turn,  shaped  the  training  content— the  extensive  list  of  suggested  DES 
topics  in  which  primary  participants  require  knowledge  and  expertise  to  produce 
desired  on-the-job  results.  The  next  section  considers  various  aspects  of  training  de¬ 
sign  that  impact  training  delivery  methods. 

TRAINING  DESIGN 

Other  considerations,  in  addition  to  the  system’s  stated  purpose,  desired  outcomes, 
competencies,  and  content,  affect  training  design.  For  example,  newly  assigned  in¬ 
dividuals  need  a  certain  amount  of  DES-specific  knowledge  immediately  to  perform 
their  jobs  effectively.  But  do  all  primary  participant  populations  need  to  know  all  of 
the  topics  within  the  body  of  knowledge  suggested  for  that  population,  or  just  a  sub¬ 
set  of  them,  to  produce  desired  results  immediately? 

In  addition,  the  frequency  of  use  of  DES-specific  knowledge,  assignment  practices 
that  result  in  high  turnover  rates  among  some  primary  participant  populations, 
widely  dispersed  primary  participant  populations,  and  the  availability  of  information 
currently  on  the  World  Wide  Web — all  related  to  the  characteristics  of  the  target 
training  population — impact  the  choice  of  training  delivery  method.  Likewise, 
learning  effectiveness  associated  with  different  delivery  methods  may  vary  (in  par¬ 
ticular,  the  effectiveness  of  self-directed  computer-based  distance  training  versus  the 
effectiveness  of  traditional  classroom  training).  These  considerations  are  addressed 
in  the  following  sections.  Cost,  another  important  consideration,  is  addressed  in 
Chapter  7. 

DES-Specific  Knowledge  Needed  Immediately  Upon  Assignment 

Every  disability  case  is  unique.  Every  disability  case  that  a  primary  participant  han¬ 
dles,  including  the  first  one,  potentially  requires  the  PEBLO,  the  physician,  the  PEB 
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member,  and  any  other  primary  participant  to  bring  to  bear  any  aspect  of  the  specific 
body  of  knowledge  and  skills  required  for  that  primary  participant  to  produce  the 
desired  on-the-job  results.  A  primary  participant  who  (1)  does  not  know  or  apply  a 
single  relevant  policy  application  standard  or  rule;  (2)  does  not  follow  a  prescribed, 
disciplined  procedure;  (3)  does  not  document  the  rationale  that  underlies  the  deci¬ 
sions;  or  (4)  does  not  focus  on  desired  system  outcomes,  invites  the  risk  of  inconsis¬ 
tent  application  of  disability  policy. 

From  a  performance  perspective,  every  primary  participant  requires  knowledge  in 
every  DBS  topic  that  constitutes  the  body  of  knowledge  for  that  particular  primary 
participant  population  before  handling  a  disability  case.  Every  case  handled  may  re¬ 
quire  knowledge  of  any  combination  of  topics  within  the  particular  body  of  knowl¬ 
edge.  However,  as  discussed  next,  not  every  primary  participant  requires  training 
immediately  upon  assignment. 

Frequency  with  Which  Primary  Participants  Apply  DES-Specific  Bodies 
of  Knowledge 

PEBLOs,  PEB  members  and  approving  authorities,  and  PEB  administrative  action  of¬ 
ficers  generally  work  full  time  in  the  DES;  therefore,  they  apply  their  specific  body  of 
knowledge  daily.  Attorneys  collocated  with  Formal  PEBs,io  who  represent  service 
members,  also  generally  apply  their  specific  body  of  knowledge  on  a  daily  basis. 
Others,  especially  PEBLOs  and  disability  evaluation  counselors,  patient  administra¬ 
tors,  Medical  Evaluation  Board  members  and  approving  authorities,  who  perform 
disability  related  tasks  as  a  collateral  duty  may  apply  aspects  of  their  specific  body  of 
knowledge  only  a  few  times  per  month.  At  the  other  end  of  the  spectrum,  physicians 
who  write  narrative  summaries  and  unit  commanders  generally  interact  sporadically 
with  the  system  and,  therefore,  apply  DES-specific  knowledge  only  sporadically  and 
infrequently.  For  example,  many  Army  unit  commanders  typically  write  no  more 
than  two  nonmedical  assessment  letters  during  their  12-  to  18-month  command. 

The  frequency  with  which  different  primary  participant  populations  apply  their  DES- 
specific  bodies  of  knowledge  influences  the  choice  of  training  delivery  method. 
Those  primary  participants  who  apply  DES-specific  knowledge  frequently  as  a  major 
part  of  their  job  stand  to  benefit  from  training  at  the  beginning  of  their  assignment  to 
a  position  in  the  DES.  Likewise,  primary  participants  who  use  this  knowledge  infre¬ 
quently  stand  to  benefit  from  “just-in-time”  training.  In  both  cases,  training  made 
available  on  an  as-needed  basis  is  an  important  consideration  in  choosing  a  delivery 
method. 


^^Attorneys  collocated  with  Formal  PEBs  do  not  work  for  the  PEB. 
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Assignment  Practices  Causing  High  Turnover  Rates  Among  Some 
Primary  Participants 

Combined  with  the  frequency  of  use  of  specific  bodies  of  DES  knowledge,  certain  as¬ 
signment  practices — such  as  selecting  individuals  with  no  DES  knowledge  and  skills 
for  primary  participant  positions  and  relatively  frequent  rotations  of  military  per¬ 
sonnel— exacerbate  the  risk  of  inconsistent  disability  policy  application.  The  assign¬ 
ments  of  some  primary  participants  are  more  stable  than  others.  For  example,  mili¬ 
tary  department  civilian  employees  assigned  as  PEBLOs  and  civilian  physicians  who 
write  narrative  summaries  and  serve  as  PEB  members  remain  in  their  positions  for 
longer  periods  than  their  military  counterparts. 

From  a  training  perspective,  these  personnel  management  policies  create  a  need  for 
more-frequent  training  delivery.  As  noted  in  the  previous  section,  training 
availability  on  an  as-needed  basis  is  an  important  consideration  in  choosing  a 
delivery  method  to  mitigate  the  effects  of  frequent  turnover. 

Widely  Dispersed  Primary  Participant  Populations 

As  noted  earlier  in  this  chapter,  physicians  are  the  most  widely  dispersed  population 
because  virtually  any  physician  in  an  MTF  may  write  a  narrative  summary.  PEBLOs, 
disability  evaluation  counselors,  patient  administrators,  and  physicians  located 
wherever  MTFs  exist  and  who  serve  on  Medical  Evaluation  Boards  are  also  widely 
dispersed. 

PEB  members,  approving  authorities,  and  administrative  action  officers  in  the  three 
military  departments  are  assigned  to  eight  separate  locations.  Attorneys  who  repre¬ 
sent  service  members  are  currently  dispersed  over  the  same  six  locations  as  the 
Formal  PEBs.  Conducting  Formal  PEBs  via  teleconferencing,  which  the  Army  is  ex¬ 
ploring,  increases  the  dispersion  of  attorneys  who  interact  with  the  DES  because  the 
technology  allows  them  to  do  their  job  from  virtually  any  military  installation.  Active 
and  Reserve  component  unit  commanders  who  interact  with  the  DES  from  poten¬ 
tially  any  military  installation  and  deployment  are  the  most-dispersed  populations. 

Although  organized  into  five  training  population  clusters,  the  dispersion  of  the  12 
primary  participant  populations  presents  yet  another  consideration  in  choosing  a 
training  delivery  method.  Widely  dispersed  target  training  populations  stand  to 
benefit  from  “transportable”  training  packages  that  are  easily  accessible  regardless  of 
the  dispersion  pattern. 

Military  Department  DES  Web  Sites 

Each  of  the  military  departments’  Disability  Evaluation  Systems  faces  the  challenges 
associated  vrith  widely  dispersed  primary  participants  on  a  daily  basis.  As  a  result, 
they  have  established  or  are  in  the  process  of  establishing  a  presence  on  the  World 
Wide  Web  as  a  valuable  resource  for  sharing  information.  Web  sites  in  various  stages 
of  development  exist  today.  The  Army  has  two  DES  Web  sites:  the  U.S.  Army  Medical 
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Command  site  at  www.armymedicine.army.mil/pad  and  the  U.S.  Army  Personnel 
Command  site  at  www.perscom.army.mil/tagd/pdapage.htm. 

The  Web  site  for  the  Air  Force  Physical  Disability  Division  can  be  found  at 
www.afpc.randolph.af.mil/disability  and  the  Department  of  the  Navy  PEB  Web 
site  is  at  wvw.hq.navy.mil/ncpb  as  of  this  writing.  Other  Web  sites  developed  by 
individual  PEBLOs  also  exist. 

Many  primary  participants  who  are  continuously  exploring  new  tools  with  which  to 
improve  the  performance  of  their  DES  have  identified  the  Web  as  an  information 
source  that  meets  many  of  their  needs.  Acceptance  of  and  comfort  with  ad  hoc 
computer-based  training  points  to  self-directed  computer-based  distance  training  as 
a  viable  training  delivery  method. 

Advantages  and  Disadvantages  of  Self-Directed  Computer-Based  Distance 
Training  and  Classroom  Training 

A  common  cultural  trait  across  all  military  departments  is  the  high  commitment  to 
excellence  in  training,  regardless  of  the  training  method.  Numerous  studies  in  the 
training  literature  report  no  significant  difference”  between  self-directed  computer- 
based  distance  learning  and  traditional  classroom  learning.  TheASTD  Training  and 
Developmen  t  Handbook  (Craig,  1996)  reports,  “Eighty-five  percent  mastery  on  a  good 
SDL  (self-directed  learning)  package  is  common,  and  95  percent  is  not  unusual.” 
And,  other  studies  report  superior  results  from  one  method  over  the  other. 

As  the  debate  on  effectiveness  of  distance  learning  continues,  Public  Law  105-261, 
enacted  on  October  17,  1998,  required  the  Secretary  of  Defense  to  develop  and  sub¬ 
mit  to  Congress  by  March  1,  1999,  a  strategic  plan  for  guiding  and  expanding 
distance-learning  initiatives  within  the  DoD,  focusing  on  the  goals  and  objectives 
of  DoD  training  and  education.  The  plan  also  provides  for  expansion  of  distance¬ 
training  initiatives  over  five  consecutive  years  beginning  with  fiscal  year  2000. 

Based  on  the  preceding  design  considerations,  we  identified  the  following  advan¬ 
tages  and  disadvantages  of  self-directed  computer-based  distance  training  and  class¬ 
room  training  for  primary  participants,  military  departments,  and  the  OSD.  This 
analysis  includes  comparisons  of  current  military  department  training  practices  and 
proposed  training  packages  developed  and  monitored  by  the  OSD. 

Primary  Participants:  Advantages  of  Self-Directed  Computer-Based  Distance 
Training 

•  Available  when  primary  participant  needs  training,  upon  assignment 

•  Tailored  to  specific  training  needs  of  each  training  population 

•  Provides  up-to-date  review  and  reference  source 

•  Primary  participant  learns  at  own  pace 

•  Immediate  feedback 

•  No  surprise  questions  or  pop  tests 
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•  Individual  chooses  order  of  material 

•  Promotes  computer  literacy 

Primary  Participants:  Disadvantages  of  Computer-Based  Distance  Training 

•  Some  not  used  to  being  self-directed  learners 

•  Some  not  comfortable  relying  on  training  objectives  instead  of  faculty 

•  Lack  of  faculty  presence 

•  Lack  of  group  synergy 

Primary  Participants:  Advantages  of  Classroom  Training 

•  Practice  applying  specific  body  of  knowledge  develops  skills  to  produce  consis¬ 
tent  determinations 

•  Peer  networking  across  military  departments  builds  collaborative  relationships 

•  Synergy  of  group 

•  Reassurance  from  adjunct  faculty 

•  Immediate  answers  to  questions 

•  Immediate  feedback 

•  Discover  others  have  same  problems 

•  Case  studies  provide  review  and  reference  source 

Primary  Participants:  Disadvantages  of  Classroom  Training 

•  Infrequent  training  schedule;  training  not  always  available  when  primary  partic¬ 
ipants  need  it 

•  Rigid  learning  schedule 

•  Time  away  from  job 

Military  Departments:  Advantages  of  Self-Directed  Computer-Based  Distance 
Training 

•  Requires  fewer  trainers/faculty 

•  Less  time  on  the  road  for  PER  members,  approving  authorities,  and  administra¬ 
tive  action  officers 

•  Reduces  trainer/faculty  travel  and  lodging  expenses 

•  Consistent  with  high-tech  military  of  the  future 

•  Just-in-time  training 

•  Up-to-date  reference  for  on-the-job  training 

•  Standardized  bodies  of  DES-specific  knowledge  for  each  primary  participant 
population  cluster 
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•  Multiple-site  training 

•  No  primary  participant  travel  and  lodging  costs 

Military  Departments:  Disadvantages  of  Self-Directed  Computer-Based  Distance 
Training 

•  Reduces  OSD  policy  interpretation  flexibility 

•  Requires  designated  training  facilitators  to  be  accessible  to  all  primary  partici¬ 
pants 

•  Requires  all  primary  participants  have  computer  access  to  Web  site 
Military  Departments:  Advantages  of  Classroom  Training 

•  Requires  less  operational  downtime  for  PEB  members,  approving  authorities, 
and  administrative  action  officers  to  prepare  and  deliver  military  department 
classroom  training 

•  Requires  fewer  military  department  trainers/faculty 

•  Reduced  training  costs 

•  Standardized  bodies  of  DES-speclfic  knowledge 

•  References  for  on-the-job  training 

Military  Departments:  Disadvantages  of  Classroom  Training 

•  Reduces  OSD  policy  interpretation  flexibility 

•  Primary  participants’  travel  and  lodging  costs 

Office  of  the  Secretary  of  Defense:  Advantages  of  Self-Directed  Computer-Based 
Distance  Training 

•  Reduces  likelihood  of  misinterpreted  OSD  policy 

•  Multiple-site  training  across  military  departments 

•  Training  consistency  and  quality 

•  Standardized  DES-specific  bodies  of  knowledge 

•  Direct  link  between  OSD  policy  and  training 

•  Just-in-time  training 

•  Consistent  with  high-tech  military  of  the  future 

•  Changes  posted  immediately  to  all  primary  participants 

•  No  primary  participant  travel  and  lodging  costs 

•  No  faculty  travel  and  lodging  expenses 

•  No  meeting-room  and  materials  costs 
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Office  of  the  Secretary  of  Defense:  Disadvantages  of  Self-Directed  Computer-Based 
Distance  Training 

•  Difficult  to  develop  properly 

•  Design  and  development  costs 

•  Development  time 

•  Requires  a  Webmaster 

•  Possible  logistics  problems  with  different  computer  systems 

Office  of  the  Secretary  of  Defense:  Advantages  of  Classroom  Training 

•  Reduces  likelihood  of  misinterpreted  OSD  policy 

•  Direct  link  between  OSD  policy  and  training 

•  Standardized  bodies  of  DES-specific  knowledge 

•  Develop  a  more  “Joint”  DES  culture 

•  Networking  among  primary  participants  from  all  military  departments  leads  to 
more-consistent  application  of  DES-specific  knowledge  and  skills 

Office  of  the  Secretary  of  Defense:  Disadvantages  of  Classroom  Training 

•  Training  consistency  and  quality  may  vary  by  adjunct  faculty 

•  Requires  adjunct  faculty 

•  Adjunct  faculty  travel  and  lodging  costs 

•  Meeting-room  and  materials  costs 

•  Design  and  development  costs 

•  Development  time 

DEPARTMENT  OF  DEFENSE  TRAINING  RECOMMENDATIONS 

Based  on  our  assessment  of  training  needs  and  the  design  considerations,  this  sec¬ 
tion  describes  our  recommendations  for  DoD  disability  evaluation  training. 

Achieving  the  outcomes  suggested  in  Chapter  3,  and  meeting  the  goal  of  consistent 
application  of  disability  policy  across  and  within  military  departments,  requires  pri¬ 
mary  participant  populations  across  the  military  departments  to  possess  common 
performance  competencies  including  specific  bodies  of  DES  knowledge  and  the  nec¬ 
essary  skills  to  produce  the  desired  on-the-job  results.  We  recommend  that  the  Office 
of  the  ASD/FMP  develop  and  monitor  knowledge-based  training  in  which  the  con¬ 
tent  focuses  on  the  suggested  list  of  DES  topics  that  collectively  constitute  a  specific 
body  of  knowledge  for  each  primary  participant  population  cluster.  We  further  rec¬ 
ommend  delivering  this  knowledge-based  training  through  a  Web  site  devoted  to 
disability  evaluation  training  and  made  accessible  to  all  primary  participants. 
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This  self-directed  computer-based  distance  training  is  a  basic  course  in  disability 
evaluation.  It  is  supplemented  with  classroom  training  that  focuses  on  applying  a 
particular  set  of  the  DES  topics  within  two  of  the  primary  participant  population 
clusters  bodies  of  knowledge  in  order  to  develop  the  skills  necessary  to  evaluate  and 
adjudicate  cases  consistently  across  and  within  the  military  departments,  a  primary 
determinant  of  consistent  application  of  disability  policy. 

Recommendations  for  Self-Directed  Computer-Based  Distance  Training 

In  order  to  deliver  standardized  training  when  it  is  needed— so-called  just-in-time 
training— to  every  primary  participant  of  the  DES,  we  recommend  that  the  OSD  es¬ 
tablish,  monitor,  and  maintain  a  Web  site  devoted  to  disability  evaluation  training.  A 
great  strength  of  this  training  delivery  method  is  that  it  allows  training  designers  to 
tailor  content  to  specific  targeted  training  populations.  In  this  case,  the  DES  topics 
that  constitute  the  body  of  knowledge  suggested  for  each  primary  participant  popu¬ 
lation  cluster  across  the  military  departments  serves  as  the  basis  for  developing  con¬ 
tent. 

Table  5.3  organizes  the  comprehensive  list  of  DES  topics  into  five  distance-training 
packages,  each  designed  to  meet  the  training  needs  of  a  particular  primary  partici¬ 
pant  population  cluster.  It  further  organizes  the  topics  roughly  in  descending  order 
of  common  training  needs  across  population  clusters,  starting  with  those  topics  re¬ 
quired  by  all  population  clusters,  and  ending  with  those  topics  required  bv  onlv  one 
cluster. 

Clearly,  all  five  population  clusters  need  knowledge  of  many  of  the  same  DES  topics, 
although  different  population  clusters  need  to  know  how  to  apply  some  topics  in 
different  ways  to  achieve  their  specific  desired  on-the-job  results.  Although  the  dif¬ 
ferent  training  packages  contain  many  of  the  same  DES  topics,  the  learning  objec¬ 
tives,  content  presentation,  and  criterion  referencing,*!  addressed  later  in  this  chap¬ 
ter,  should  match  the  specific  job  application  needs  of  each  target  population  cluster 
(and  some  will  be  the  same). 

The  full  range  of  information  included  in  some  DES  topics  appears  in  italics  under 
each  shaded  area  in  Table  5.3.  Note,  in  particular,  that  not  all  primary  participant 
populations  require  knowledge  of  all  the  DES  topics. 


Criterion  referencing  refers  to  the  method  of  testing  that  is  most  often  used  in  self-directed  computer- 
based  distance  training.  The  test  questions  are  written  directly  from  the  stated  learning  objectives  and  can 
be  answered  directly  from  the  material  presented.  In  other  words,  criterion-referenced  tests  contain  no 
nidden  meanings  or  trick  questions. 


Table  5.3 

Disability  Evaluation  System  Topics  Sorted  According  to  Population-Cluster  Training  Packages 
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The  DES  topics  in  each  of  the  proposed  training  packages  target  a  particular  primary 
participant  population  cluster  to  enable  the  primary  participants  within  the  cluster 
to  produce  the  desired  on-the-job  results.  A  description  of  each  proposed  training 
package  follows: 

Training  Package  1  is  designed  for  Population  Cluster  1,  which  is  composed  of 
PEBLOs  and  disability  evaluation  counselors,  patient  administrators  who  support 
PEBLOs,  and  PEB  administrative  action  officers  who  quality-check  and  move  medi¬ 
cal  boards  to  the  appropriate  recipients  (to  physicians  for  more  information  or  an 
appropriate  signature  or  to  the  PEB).  PEBLOs  require  expertise  in  the  greatest  num¬ 
ber  of  DES  topics  to  produce  desired  results,  and  although  patient  administrators 
and  PEB  administrative  action  officers  require  expertise  in  fewer  DES  topics,  their 
needs  are  similar  enough  to  PEBLOs  to  merit  a  common  distance-training  package. 
PEBLOs  and  disability  evaluation  counselors  will  simply  complete  all  modules, 
whereas  patient  administrators  and  PEB  administrative  action  officers  will  complete 
only  those  modules  relevant  to  producing  their  desired  on-the-job  results. 

Training  Package  2  is  designed  for  Population  Cluster  2,  composed  of  referring 
physicians — those  who  write  narrative  summaries.  Medical  Evaluation  Board  mem¬ 
bers,  and  Medical  Evaluation  Board  approving  authorities.  All  three  populations 
within  this  cluster  require  expertise  in  the  same  set  of  DES  topics,  except  for  two. 
Only  the  physician  who  interacts  with  the  service  member  needs  to  know  the  differ¬ 
ences  between  DoD  and  VA  disability  systems  so  that  he  or  she  does  not  verbally 
misrepresent  any  differences  to  the  service  member  and,  therefore,  create  unrealistic 
expectations  on  the  part  of  the  member.  12  Likewise,  the  referring  physician  does  not 
need  to  know  how  to  write  documentation  to  support  the  Medical  Evaluation  Board 
decision. 

Training  Package  3  is  designed  for  Population  Cluster  3,  composed  of  PEB  members 
and  approving  authorities  and  post-PEB  level  appellate  review  authorities.  PEB 
members  emphasized  the  need  for  appellate  review  board  members  who  are  beyond 
the  Formal  PEB  to  know  all  of  the  standards,  rules,  and  other  considerations,  and  the 
prescribed,  disciplined  procedures  that  are  applied  throughout  case  adjudication. 
Otherwise,  it  is  difficult  for  the  appellate  review  board  members  to  make  seasoned 
judgments.  Our  analysis  indicated  that  attorneys  who  advise  and  represent  service 
members  during  an  appeal  to  the  Formal  PEB  require  knowledge  of  the  same  set  of 
DES  topics,  as  the  details  pertain  to  an  individual  case.  Although  this  training 
package  is  not  designed  explicitly  for  attorneys,  it  closely  matches  their  needs; 
therefore,  they  are  included  in  this  population  cluster. 

Training  Package  4,  the  smallest  of  the  five  packages,  is  designed  for  Population 
Cluster  4,  composed  solely  of  active  component  unit  commanders.  It  contains  DES 
topics  of  which  active  component  unit  commanders  require  knowledge  to  produce 
desired  results  when  they  interact  with  the  DES.  These  topics  include  how  to  prepare 


l^Although  it  is  not  the  physician’s  role  to  counsel  the  service  member  on  either  the  DoD  or  VA  system, 
several  primary  participants  reported  that  physicians  often  say  things  to  service  members  that  create 
unrealistic  expectations. 
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and  submit  a  nonmedical  assessment  (commander’s  letter)  and  an  LOD  determina¬ 
tion  on  a  service  member.  It  also  includes  topics  related  to  the  differences  between 
the  DoD  and  VA  disability  systems  so  that  the  unit  commander  can  avoid  verbally 
misrepresenting  any  differences  to  the  service  member  and,  therefore,  create  unreal¬ 
istic  expectations. 

Training  Package  5  is  designed  for  Population  Cluster  5,  composed  solely  of  Reserve 
component  commanders.  It  contains  a  blend  of  DBS  topics  of  which  PEBLOs  and 
active  component  unit  commanders  must  possess  knowledge  to  accomplish  their 
desired  results.  It  includes  many  DBS  topics  otherwise  associated  with  PEBLO 
population,  in  particular,  topics  that  are  intended  to  enable  the  Reserve  component 
commander  to  provide  accurate  information  to  the  reservist  who  has  limited  access 
to  a  PEBLO. 

We  identified  no  need  for  DoD  disability  evaluation  training  for  the  larger  personnel 
community  that  administratively  returns  the  service  member  to  duty  or  out-pro- 
cesses  and  issues  orders  for  those  separated  or  retired  for  disability. 

If  a  later  assessment  of  training  effectiveness  demonstrates  that  these  training  pack¬ 
ages  do  not  enable  the  primary  participant  populations  to  produce  the  desired  on- 
the-job  results,  the  OSD  should  modify  the  training  packages  so  that  they  perform  as 
intended. 

Development  of  the  actual  self-directed  computer-based  distance-training  curricu¬ 
lum  is  beyond  the  scope  of  this  study.  Nevertheless,  we  outline  a  developmental  ap¬ 
proach  in  the  context  of  our  other  recommendations  later  in  this  chapter. 
Development  of  computer-based  distance-training  curriculum  entails  four  critical 
factors:  job  analysis,  learning  objectives,  training  content,  and  criterion  referencing 
(Craig,  1996). 

Job  analysis  is  critical  to  identifying  all  of  the  required  tasks  (including  the  knowledge 
required  to  accomplish  those  tasks)  for  each  primary  participant  population  to  pro¬ 
duce  the  desired  on-the-job  results.  Thorough  job  analysis  of  each  primary  partici¬ 
pant  population  is  the  critical  first  step  to  developing  a  self-directed  computer-based 
distance-training  package.  The  product  of  job  analyses  forms  the  basis  for  writing 
learning  objectives  for  each  training  population  cluster. 

Writing  learner-centered  objectives,  each  directly  related  to  the  required  body  of 
knowledge  and  skills  identified  in  the  primary  participant  population  job  analysis,  is 
the  second  critical  step  in  developing  a  self-directed  computer-based  distance- 
training  package.  Learning  objectives  take  the  place  of  faculty  in  the  classroom;  they 
guide  learners  through  the  training  package  by  communicating  what  is  most  impor¬ 
tant  and  what  the  learner  must  master.  Developing  content  that  relates  directly  to 
the  learning  objectives  and  that  presents  information  in  sufficient  detail  for  the 
learner  to  master  the  objectives  is  the  third  critical  step.  Content,  written  as  concisely 
as  possible,  includes  sufficient  detailed  information  for  the  learner  to  master  the 
knowledge  and  skills  required  to  achieve  the  learning  objectives. 
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In  many  instances,  the  five  training  packages  contain  the  same  learning  objectives 
and  content.  In  other  cases,  different  population  clusters  require  different  learning 
objectives  pertaining  to  the  same  DES  topics  as  they  relate  to  jobs  within  specific 
primary  participant  population  clusters.  For  example,  the  physicians’  training  pack¬ 
age  (Package  2)  content  includes  information  in  sufficient  detail  by  specialty— in 
particular,  the  five  specialties  that  represent  the  majority  of  consults— to  enable 
physicians  to  write  sufficiently  detailed  and  complete  narrative  summaries  and  spe¬ 
cialty  consults.  The  physicians’  training  package  also  specifies  the  special  tests  and 
measurements  that  they  must  include  for  the  PEB  to  determine  a  VASRD  rating. 
Similarly,  the  unit  commanders’  training  package  (Package  4)  includes  enough  de¬ 
tailed  information  for  commanders  to  write  sufficiently  detailed  and  complete 
nonmedical  assessment  letters  and  prepare  sufficiently  detailed  and  complete  LOD 
determinations. 

The  various  training  packages’  content  may  also  include  sample  documents  required 
to  process  cases  through  the  DES,  such  as  well-written  and  sufficiently  detailed  and 
complete  narrative  summaries,  specialty  consults,  and  nonmedical  assessments. 

Last,  writing  test  questions  directly  from  the  learning  objectives  that  learners  can  an¬ 
swer  directly  from  the  content  presented— that  is,  criterion  referencing— is  the  fourth 
critical  step  in  developing  self-directed  computer-based  distance  learning.  Self- 
directed  learning  is  often  considered  a  mastery  process  because,  when  executed 
properly,  learners  know  what  is  expected  of  them,  find  what  is  required  in  an 
efficient  manner,  and  answer  test  questions  and  demonstrate  mastery  based  on  the 
content  presented  in  the  training  package. 

The  design  parameters  of  these  five  just-in-time  training  packages  extend  their  utility 
beyond  pure  training  materials  to  serving  as  an  up-to-date  and  comprehensive  desk¬ 
top  reference  tailored  to  the  needs  of  each  population  cluster.  For  example,  both  the 
physicians’  and  unit  commanders’  training  packages  (Packages  2  and  4)  ideally  in¬ 
clude  embedded  links  to  electronic-based  templates  that  enable  them  to  write  and 
electronically  submit  narrative  summaries  and  specialty  consults,  or  nonmedical  as¬ 
sessments  (commanders’  letters).  Links  to  a  set  of  frequently  asked  questions  of  po¬ 
tential  interest  to  all  primary  participants,  cohorts,  and  centrally  located  experts,  and 
links  to  all  of  the  DES  references  including  relevant  statutes  and  OSD  and  military 
department  policy  documents  extend  the  value  of  the  training  packages. 

This  self-directed  computer-based  distance-training  method  benefits  from  on-site 
facilitators  to  guide  learners  through  the  distance-learning  process.  Very  experienced 
peers  or  supervisors  may  serve  as  facilitators,  and  facilitators  may  certify  satisfactory 
completion  of  individual  training  goals  and  objectives. 

Section  4.4.5  of  DoD  Directive  1332.18  assigns  responsibility  to  the  Secretaries  of  the 
military  departments  to  ensure  that  PEB  members  and  applicable  review  authorities 
are  trained  and  certified  in  physical  disability  evaluation.  Well-designed  self-directed 
computer-based  training  is  a  natural  for  use  as  a  certification  tool,  particularly  self- 
certification.  This  approach  to  certification  is  a  matter  of  (1)  constructing  learning 
objectives,  content,  and  test  questions  that  meet  desired  certification  standards  for 
the  target  population,  (2)  building  in  an  honor  system,  and  (3)  designing  the  training 
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package  for  each  target  population  cluster  such  that  primary  participants  in  that 
population  cluster  can  self-certify  their  mastery  of  required  knowledge  and  skills. 
Supervisors  may  also  choose  to  certify  a  primary  participant’s  competence  by  com¬ 
bining  the  self-certification  test  results  with  observed  job  performance. 

One  final  point  on  self-directed  computer-based  distance  training:  Selecting  experi¬ 
enced  and  credible  subject-matter  experts  and  technical  designers  who  are  able  to 
develop  high-quality  training  that  produces  the  desired  on-the-job  results  from  the 
populations  trained  is  also  critically  important.  The  subject-matter  experts  actually 
serve  as  adjunct  faculty  who  are  delegated  authority  by  the  OSD  to  develop  the  DoD 
disability  evaluation  training. 


Recommendations  for  Classroom  Training 

At  this  point,  we  assume  that  the  OSD  has  developed  a  Web  site  devoted  to  disability 
evaluation  training  and  has  established  the  recommended  self-directed,  computer- 
based  distance-training  packages.  We  further  assume  the  training  packages  “teach” 
the  DES  topics  to  the  degree  intended.  Now,  the  question  is,  do  primary  participants 
require  additional  training  to  apply  policy  consistently  across  and  within  military  de¬ 
partments  to  produce  the  desired  on-the-job  results? 

To  answer  this  question,  we  reexamined  the  proposed  primary  participant  compe¬ 
tencies  presented  earlier  in  this  chapter.  The  competency  statements  suggest  that 
PEB  members,  PEB  approving  authorities,  and  post-PEB  appellate  review  board 
members  across  military  departments,  in  particular,  stand  to  benefit  from  collabora¬ 
tion  with  peers  on  how  to  uniformly  apply  the  rules,  procedures,  and  other  consider¬ 
ations  in  determining  fitness,  assigning  the  VASRD  or  analogous  codes,  and  deter¬ 
mining  disability  ratings. 

Likewise,  Medical  Evaluation  Board  members  and  approving  authorities  across  mili¬ 
tary  departments  stand  to  benefit  from  collaboration  with  peers  regarding  how  to 
apply  disciplined  medical  retention  standards  uniformly  such  that  members  having  a 
similar  condition  and  similar  office,  grade,  rank,  or  rating  receive  similar  medical  re¬ 
tention  decisions.  Attorneys  are  also  likely  to  benefit  from  classroom  training,  with 
proof  of  distance-learning  certification,  if  classroom  space  is  available. 

Although  PEBLOs  and  disability  evaluation  counselors  require  expertise  in  all  of  the 
DES  topics  recommended  for  classroom  training,  and  some  may  wish  to  extend  their 
expertise  to  this  higher  level  required  for  adjudicating  cases,  the  disability  evaluation 
classroom  training  recommended  here  is  not  targeted  to  their  specific  performance 
needs.  Rather,  the  classroom  training  is  intended  to  develop  the  skills  necessary  for 
the  PEB  members  and  approving  authorities  and  post-PEB  appellate  review  board 
members  to  evaluate  and  adjudicate  cases  in  a  consistent  manner  across  the  military 
departments  and  for  the  Medical  Evaluation  Board  members  and  approving 
authorities  to  apply  disciplined  medical  retention  standards  uniformly  such  that 
members  having  a  similar  condition  and  similar  office,  grade,  rank,  or  rating  receive 
similar  medical  retention  decisions.  Satisfactory  classroom  performance  can  also 
serve  as  another  aspect  of  certification. 
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Note  that  active  and  Reserve  component  commanders  are  not  included  in  the  class¬ 
room  training  because  our  analysis  suggests  that  they  benefit  sufficiently  from  their 
self-directed  computer-based  distance-training  packages. 

In  addition,  numerous  primary  participants  that  we  interviewed  expressed  concern 
about  the  difficulty  both  Medical  Evaluation  Board  members  and  PEB  members  had 
in  applying  the  many  standards,  rules,  procedures,  and  other  considerations  consis¬ 
tently  within  military  departments,  let  alone  across  military  departments.  Three  is¬ 
sues  of  concern  surfaced  repeatedly  during  our  interviews  that  reinforce  the  need  for 
more-advanced  classroom  training  to  enable  physicians  and  nonmedical  PEB 
members,  in  particular,  to  apply  disability  policy  consistently  within  and  across  mili¬ 
tary  departments.  The  issues,  as  articulated  most  clearly  and  concisely  by  primary 
participants,  are  as  follows: 

1.  Human  elements  such  as  emotions  and  personality  traits,  “good  service  member” 
versus  “bad  service  member”  issues,  and  the  member’s  length  of  service— espe¬ 
cially  when  nearing  20  years — hamper  efforts  to  render  fair  and  consistent  de¬ 
cisions. 

2.  PEB  decisions  change  noticeably  with  new  members’  differing  personal  philos¬ 
ophies. 

3.  Primary  participants  interpret  DoD  policy  and  apply  it  consistently  to  the  best  of 
their  ability.  However,  primary  participants  do  not  converse  with  their 
counterparts  from  the  other  military  departments  or  the  OSD  so  they  have  no  way 
of  knowing  if  they  are  passing  judgments  that  differ  from  their  counterparts  in  the 
other  military  departments. 

As  a  result,  we  recommend  supplementing  the  DoD  self-directed  computer-based 
distance-training  packages  with  DoD  classroom  training  for  PEB  members  and  ap¬ 
proving  authorities,  appellate  review  board  members,  and  Medical  Evaluation  Board 
members  and  approving  authorities  across  the  military  departments.  The  classroom 
training  focuses  on  applying  a  particular  set  of  DES  topics  to  develop  the  skills  neces¬ 
sary  to  evaluate  and  adjudicate  cases  and  apply  disability  policy  consistently  across 
and  within  the  military  departments. 

The  classroom  training  is  designed  explicitly  to  supplement  the  self-directed  com¬ 
puter-based  distance  training.  As  such,  completing  the  appropriate  distance-training 
package  is  a  prerequisite  for  enrolling  in  classroom  training,  evidenced  perhaps  by  a 
certificate  of  self-mastery  of  the  required  knowledge  and  skills.  Learning  objectives, 
content,  and  student  learning  evaluation  differ  from  the  distance-training  package  in 
that  they  focus  on  applying  a  particular  set  of  the  DES  topics  learned  in  the  distance¬ 
training  packages  to  a  variety  of  real-life  cases. 

We  refined  the  comprehensive  list  of  DES  topics,  which  are  presented  in  Appendix  G, 
to  a  shorter  list  of  DES  topics  that  form  the  basis  for  classroom-training  content,  as 
shown  in  Table  5.4.  This  set  of  DES  topics  enables  students  to  practice  applying  the 
numerous  standards,  rules,  procedures,  and  other  considerations  to  a  wide  variety  of 
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case  studies  in  a  controlled  classroom  environment  in  which  students  collaborate  on 
making  decisions  that  result  in  consistent  dispositions. 

The  letters  A  and  B  in  Table  5.4  are  used  to  represent  two  logical  training  modules, 
one  developed  to  present  to  primary  participant  population  Cluster  2  and  Cluster  3,13 
and  the  other  presented  only  to  population  Cluster  3.  The  full  range  of  information 
included  within  some  DES  topics  appears  in  italics  under  the  shaded  entries. 

Smaller  class  sizes  are  preferable  for  practicing  case  evaluation  and  adjudication. 
Given  the  pent-up  need  for  this  training  and  the  frequency  of  primary  participant 
turnover,  we  envision  scheduling  classroom  training  quarterly  during  the  first  and 
second  years,  and  determining  frequency  for  the  outlying  years  based  on  the  needs  at 
the  time. 

As  with  the  self-directed  computer-based  distance  training,  selecting  experienced 
and  credible  subject-matter  experts  who  are  able  to  develop  and  deliver  high-quality 
training  that  produces  the  desired  on-the-job  results  from  the  populations  trained  is 
critically  important.  And  as  with  distance  training,  these  subject  matter  experts  serve 
as  adjunct  faculty  who  are  delegated  the  authority  by  the  OSD  to  develop  and  deliver 
the  DoD  disability  evaluation  training. 

And,  like  the  recommended  computer-based  distance-training  packages  presented 
in  Table  5.2,  this  classroom-training  package  is  based  on  the  system  purpose  and  de¬ 
sired  outcomes  that  inform  the  primary  participant  competencies  proposed  in  this 
report.  Also,  like  the  distance-training  packages,  this  classroom  training  package  is  a 
template,  or  starting  point,  for  consideration  by  the  Office  of  the  ASD/FMP,  in  con¬ 
sultation  with  the  ASD/HA  and  the  ASD/RA,  and  representatives  of  the  military  de- 
partments'  PEBs  and  Office  of  the  Surgeons  General. 

If  further  OSD  consultations  with  experienced  DES  experts  result  in  changes  to  the 
purpose,  desired  outcomes,  and/or  primary  participant  competencies  that  serve  to 
inform  specific  training  content  and  packages,  or  changes  to  the  content  itself,  the 
Office  of  the  ASD/FMP  should  consider  the  merit  of  the  changes  and  make  the  ap¬ 
propriate  alterations.  If  a  later  assessment  of  training  effectiveness  demonstrates  that 
PEBLOs  or  other  primary  participants  require  classroom  training,  then  the  Office  of 
the  ASD/FMP  should  modify  the  training  packages  to  enable  the  primary  partici¬ 
pants  to  produce  the  desired  on-the-job  results. 

Although  training  is  often  viewed  as  a  cost  of  doing  business,  it  should  more  appro¬ 
priately  be  viewed  as  an  investment.  Training  that  enables  each  primary  participant 
of  the  DES  to  produce  desired  on-the-job  results  using  the  knowledge  and  skills 
learned  produces  a  return  on  that  investment.  Participants  who  continue  to  use  the 
knowledge  and  skills  they  acquire  through  training  represent  a  return  that  grows  in 
value. 


X/,  participant  population  Cluster  2  includes  physicians  who  write  narrative  summaries  and 

Medical  Evaluation  Board  members  and  approving  authorities.  Primary  participant  population  Cluster  3 
includes  PEB  members  and  approving  authorities,  post-PEB  appellate  review  board  members,  and 
attorneys. 


Table  5.4 

Disability  Evaluation  System  Topics  Forming  the  Basis  for  Classroom  Training  Modules 
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Table  5.4 — Continued 
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In  the  next  chapter,  we  describe  a  suggested  structure  for  a  management  information 
system  capable  of  monitoring  DES  performance  (at  the  DoD,  military  department, 
and  MTF  levels),  and  Chapter  7  presents  a  cost-benefit  analysis  of  the  training 
intervention  recommended  here. 


Chapter  Six 

MONITORING  SYSTEM  PERFORMANCE 


Of  the  ten  recommended  interventions  listed  in  Chapter  4,  two  have  the  greatest  im¬ 
pact  on  system  performance.  One  of  them  is,  of  course,  training.  The  other  is  the  de¬ 
velopment  of  a  management  information  system.  As  stated  previously  in  this  report, 
we  recommend  that  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and  the 
ASD/RA,  direct  the  Director  of  Officer  and  Enlisted  Personnel  Management  to  de¬ 
velop  and  maintain  a  comprehensive  management  information  system  that  is  capa¬ 
ble  of  monitoring  relevant  performance  measures. 

Monitoring  key  performance  measures  (as  they  apply  to  hoth  the  active  and  Reserve 
components)  enables  the  OSD  and  military  department  DES  leaders  to  assess,  ana¬ 
lyze,  and  take  action  to  continually  improve  the  performance  of  the  DES.  Regularly 
reporting  system  performance  to  senior  military  department  officials  and  the  OSD  in 
a  valid  and  meaningful  way  is  essential  to  the  DES  primary  participants  being  able  to 
carry  out  their  responsibilities.  This  chapter  outlines  the  specifications  for  just  such  a 
performance-monitoring  system. 

The  military  departments  operate  their  DESs  for  the  benefit  of  two  customer  groups: 
the  service  members  and  the  individual  military  services.  DES  performance  should 
be  judged  by  how  well  it  meets  the  expectations  of  these  external  customers.  In  turn, 
a  stated  system  purpose  and  desired  outcomes  define  customer  expectations.  Service 
member  expectations  center  on  similar  dispositions  for  service  members  in  similar 
circumstances  and  on  due  process,  whereas  military  service  expectations  center  on 
expeditious  processing  of  disability  cases  and  efficient  DES  operations. 

We  recommend  that  the  OSD  monitor  a  variety  of  performance  measures.  These 
measures  encompass  (1)  perceptions  that  come  directly  from  customers  on  how  well 
the  DES  meets  their  expectations  and  (2)  indirect,  but  more  objective  and  quantita¬ 
tive,  measures.  The  latter  measures  primarily  include  outcome  and  output  measures 
augmented  with  one  input  measure^  linked  together  in  a  framework  that  identifies 
the  relationships  among  the  measures  and  how  they  affect  overall  performance  of 
the  DES.  Outcome  measures  include  case  variability,  number  of  decisions  appealed 


^Outcome  measures  assess  the  results  of  a  program  or  system  compared  with  its  intended  purpose;  they 
are  lagging  indicators  of  performance.  These  measures  represent  the  quantitative  manifestation  of 
customer  expectations.  Output  measures  assess  immediate  performance  results  of  key  parts  of  a  system 
that  contribute  to  system  outcomes.  They  are  a  mix  of  lagging  and  leading  indicators  of  performance, 
where  each  indicator  is  linked  to  others  in  a  logical  way. 


Ill 
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by  service  members,  amount  of  time  to  replace  an  unfit  service  member,  and  total 
system  cost.  Output  meusures  include  percentage  of  primary  participants  certified, 
productivity,  cost  per  case,  average  processing  time,  number  of  reworks,  and  amount 
of  time  to  promulgate  policy  change.  We  suggest  one  input  measure,  total  resources 
devoted  to  the  DES.  We  suggest  multiple  metrics  for  each  performance  measure  later 
in  this  chapter.^ 

To  motivate  the  military  departments  to  improve  system  performance,  we  recom¬ 
mend  that  the  OSD  benchmark  metrics  against  trends,  an  OSD  standard,  and/or 
DoD  averages.  In  addition,  using  the  framework  underlying  the  performance  mea¬ 
sures,  the  OSD  should  establish  targets  for  key  metrics  and  require  the  military  de¬ 
partments  to  present  a  plan  for  how  they  are  going  to  achieve  the  OSD  targets  over  a 
specified  time  period,  followed  by  a  report  of  the  military  department’s  actual  met¬ 
rics  at  the  end  of  the  period. 

Each  military  department  plan  should  delineate  specific  actions  to  achieve  the  OSD 
key  metric  targets  and  its  DES  performance  objectives  for  the  specified  period.  Each 
military  department  performance  report  should  communicate  its  performance 
(metrics)  on  the  specified  measures  and  its  own  performance  objectives  to  the  OSD 
at  the  end  of  the  specified  period. 

Finally,  we  recommend  summarizing  the  information  that  is  gathered  and  analyzed 
and  feeding  it  back  to  the  primary  participants  of  the  DES. 

To  be  most  effective,  performance  monitoring  must  be  results-oriented  (and  not  fo¬ 
cused  on  inputs  or  processes),  built  into  the  operational  routines  of  the  system,  and 
reported  frequently  and  publicly.  On  the  other  hand,  monitoring  performance  is  a 
waste  of  time  if  the  information  received  is  not  evaluated  and  acted  upon.  The  man¬ 
agement  information  system  recommended  in  this  chapter  provides  the  foundation 
for  system  performance  evaluation  and  leadership  action  to  establish  such  a 
management  information  system. 

THE  NEED  FOR  A  MANAGEMENT  INFORMATION  SYSTEM 

A  management  information  system  is  essential  to  effectively  carrying  out  the  respon¬ 
sibilities  assigned  by  DoD  Instruction  1332.38.  As  the  primary  vehicle  for  providing 
feedback,  this  system  is  also  critical  to  the  success  of  training  and  other  interventions 
recommended  in  previous  chapters  of  this  report. 

Although  DoD  Instruction  1332.38  does  not  specifically  mandate  a  management  in¬ 
formation  system  to  carry  out  its  directives,  it  does  assign  responsibilities  we  believe 
can  only  be  effectively  accomplished  with  input  from  such  a  system.  In  particular. 


2 

The  terms  we  employ  in  this  report  form  a  hierarchical  structure  of  increasing  specificity.  We  start  with 
the  term  outcomes.  Outcomes  are  the  system  results  of  importance  to  external  customers.  We  define  them 
m  terms  of  custorner  expectations.  Outcome  measures  represent  the  quantitative  manifestation  of  these 
expectations.  Similarly,  output  measures  represent  the  quantitative  manifestation  of  intermediate  results 
produced  by  the  system.  Metrics  are  more-quantitative  representations  of  outcomes,  outputs,  and 
inputs  the  actual  data  in  the  management  information  system. 
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Section  5.2.3  of  DoD  Instruction  1332.38  directs  the  ASD/FMP  to  “Establish 
necessary  reporting  requirements  to  monitor  and  assess  the  performance  of  the  DES 
and  compliance  of  the  military  departments  with  [DoDI  1332.38]  and  DoD  Directive 
1332.18.”  Section  5.2.7  further  directs  the  ASD/FMP  to  “Develop  quality  assurance 
procedures  to  ensure  that  policies  are  applied  in  a  fair  and  consistent  manner.” 
Section  5.5.2  directs  the  Secretaries  of  the  military  departments  to  “Establish  a  qual¬ 
ity  assurance  process  to  ensure  that  policies  and  procedures  established  by  [DoDD 
1332.18  and  DoDI  1332.38]  are  interpreted  uniformly.” 

Unfortunately,  the  data  required  to  fulfill  these  responsibilities  are  not  currently 
available.  The  OSD  gathers  little  data  on  the  operation  of  the  DES;  the  limited  data 
focus  on  the  amount  of  time  taken  to  process  medical  boards  and  on  the  types  of  dis¬ 
abilities  that  are  processed.  The  information  collected  is  practically  useless  in  assess¬ 
ing  consistency  of  policy  application.  Among  the  military  departments,  the  Army 
collects  and  analyzes  the  greatest  amount  of  data;  we  applaud  its  plan  to  expand  its 
data  collection  and  analysis.  Nevertheless,  this  is  a  situation  that  needs  to  be  cor¬ 
rected  at  the  OSD  level  if  senior  leadership  is  to  meet  its  responsibilities  for  ensuring 
the  consistent  application  of  disability  policy. 

In  addition  to  monitoring  and  assessing  the  degree  to  which  the  DES  accomplishes 
its  purpose  while  potentially  serving  as  the  basis  for  future  interventions  to  improve 
performance,  a  management  information  system  is  needed  to  assess  the  effective¬ 
ness  of  the  interventions  we  recommend  in  Chapter  4. 

The  analysis  of  the  issues  we  identified  through  interviews  with  primary  participants 
and  by  attending  major  military  department  training  events  served,  in  effect,  as  a 
surrogate  for  a  more  desirable  and  comprehensive  analysis  that  would  have  been 
possible  had  a  management  information  system  that  monitors  outcome  and  output 
measures  been  in  place.  Although  this  analysis  was  an  effective  mechanism  for  iden  ¬ 
tifying  the  most-obvious  areas  of  concern,  it  does  not  allow  for  feedback  with  which 
to  judge  the  effectiveness  of  the  recommended  interventions.  Because  the  recom¬ 
mended  interventions  are  based  on  information  that  is  not  necessarily  complete, 
objective,  and  empirically  based,  we  expect  that  the  interventions  are  not  as  finely 
tuned  as  they  otherwise  might  be. 

A  department-wide  management  information  system  can  support  the  need  for  in¬ 
formation  to  carry  out  the  leadership  responsibilities  both  in  the  OSD  and  in  the 
military  departments  and  evaluate  and  further  tailor  the  recommended  interven¬ 
tions.  This  chapter  outlines  the  characteristics  of  such  a  management  information 
system. 

The  following  sections  of  this  chapter  focus  on  the  question,  how  should  the  leader¬ 
ship  of  the  DES  assess  the  system’s  performance?  The  chapter  then  describes  a 
framework  relating  major  parts  of  the  DES  that,  individually  and  in  combination,  in¬ 
fluence  system  performance.  Finally,  in  the  context  of  this  framework,  this  chapter 
delineates  a  set  of  specific  metrics  to  collect,  organize,  and  analyze  in  order  to  assess 
system  performance. 
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ASSESSING  THE  PERFORMANCE  OF  THE  DISABILITY 
EVALUATION  SYSTEM 

A  management  information  system  can  provide  data  in  many  forms.  The  data  of 
greatest  interest  in  the  context  of  this  report  are  those  that  help  the  ASD/FMP,  the 
ASD/HA,  and  the  ASD/RA  assess  DES  performance.  Generally,  a  system’s  perfor¬ 
mance  is  best  evaluated  from  the  perspective  of  its  external  customers.  Most  of  the 
primary  participants  with  whom  we  spoke  identified  two  primary  external  cus¬ 
tomers:  the  service  members  and  the  military  services. 

Although  primary  participants  identified  service  members  as  customers  of  the  DES, 
some  suggested  that  service  members  abuse  the  system  by  manipulating  its  proce¬ 
dures  to  delay  their  return  to  duty  or  to  maximize  compensation  for  injury  or  disease. 
Although  these  kinds  of  behaviors  undoubtedly  exist,  primary  participants  expressed 
confidence  that  the  vast  majority  of  service  members  just  want  the  system  to  treat 
them  fairly. 

Indeed,  from  the  military  service’s  perspective,  the  basic  purpose  of  the  DES  is  to 
evaluate  and  remove  service  members  who  are  unable  to  perform  their  duties  so  that 
unit  commanders  may  requisition  replacements  who  are  able  to  perform  those  du¬ 
ties.  In  the  broadest  sense,  the  DES  primarily  serves  the  higher-level  departmental 
objective  of  maintaining  a  fit  and  ready  force. 

For  the  task  of  designing  a  management  information  system,  the  stated  DES  purpose 
serves  as  a  statement  of  external  customer  expectations  regarding  system  perfor¬ 
mance.  As  noted  in  Chapter  3,  we  formulated  the  following  proposed  statement  of 
purpose  of  the  DES:  to  evaluate  service  members  with  potentially  unfitting  condi¬ 
tions  in  a  fair,  consistent,  efficient,  and  timely  manner  and,  likewise,  to  remove  those 
unable  to  fulfill  the  duties  of  their  office,  grade,  rank,  or  rating  and  determine  a 
disability  rating  percentage  for  those  removed.  Chapter  3  also  delineates  five 
outcomes  that  elaborate  on  the  stated  purpose. 

Table  6.1  portrays  the  relationship  between  the  outcomes  for  the  DES  and  customer 
expectations  for  those  outcomes. 


Table  6.1 


Disability  Evaluation  System  Outcomes  and  Customer  Expectations 


Outcomes 

Expectations 

Service  members  having  a  similar  condition  and  similar  office,  grade, 
rank,  or  rating  receive  similar  fitness  decisions  within  the  military 
department 

Similar  dispositions 

Service  members  found  unfit  receive  similar  disability  ratings  for  similar 
conditions  across  and  within  the  military  departments 

Similar  dispositions 

Service  members  freely  and  appropriately  exercise  their  rights  to 
administrative  due  process 

Due  process 

Service  members  return  to  duty,  or  separate  or  retire  for  disability  in  a 
timely  manner 

Expeditious  processing 

Primary  participants  perform  their  duties  as  efficiently  as  possible  so  that 
collectively  they  return  service  members  to  duty,  or  separate  or  retire 
service  members  in  a  fair,  consistent,  and  timely  manner 

Efficient  operations 
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The  two  customer  categories  (service  members  and  military  services)  align  nicely 
with  these  customer  expectations.^  In  particular,  fair  and  consistent  treatment  pri¬ 
marily  reflects  how  well  the  DBS  fulfills  the  expectations  of  service  members.  This 
means  the  system  fulfills  service  member  expectations  when 

•  service  members  in  similar  circumstances  with  similar  conditions  receive  similar 
dispositions  (fitness,  disability  rating,  and  personnel  actions) 

•  service  members  are  afforded  administrative  due  process  (namely,  they  receive 
the  information  they  need  to  make  informed  decisions  and  to  evaluate  the  de¬ 
cisions  rendered  by  the  system,  and  they  receive  the  opportunity  and  resources 
for  appeal  to  address  reasonable  concerns). 

Expeditious  processing  and  efficient  operations  primarily  reflect  how  well  the  DES 
fulfills  the  expectations  of  the  services.  This  means  the  system  fulfills  service  expec¬ 
tations  when 

•  primary  participants  process  medical  boards  according  to  explicit  time  standards 

•  primary  participants  utilize  “best  business  practices”  to  meet  standards  and 
honor  service  member  expectations. 

The  OSD  and  the  military  departments  can  and  should  assess  system  performance 
directly  in  terms  of  how  well  service  members,  commanders,  and  other  senior 
leaders  perceive  that  the  system  meets  their  expectations.  In  particular,  audit  and 
survey  instruments  (such  as  Inspector  General  reviews  and  OSD  surveys)  can  assess 
service  members’  perceptions  of  fairness  and  due  process. 

Service  members’  perceptions  are  naturally  inherent  in  each  service  member;  the  ac¬ 
curacy  or  validity  of  the  perceptions  does  not  necessarily  affect  whether  the  service 
member  believes  the  system  fulfilled  his  or  her  expectations.  Nevertheless,  these 
audit  and  survey  instruments  can  inquire  about  sources  of  satisfaction  and  dissatis¬ 
faction  with  the  system  as  it  relates  to  the  service  member’s  expectations.^  Similarly, 
the  OSD  can  assess  how  well  the  DES  fulfills  the  expectations  of  the  services  by  ob¬ 
serving  the  numerous  mechanisms  (personal  communications,  leadership  direction 
documents,  budgets,  and  other  information)  that  commanders  and  other  senior 
leaders  use  to  express  formally  and  informally  their  satisfaction  or  dissatisfaction.^ 


^This  is  not  to  suggest  that  the  two  types  of  customers  are  concerned  solely  with  their  respective 
expectations,  but  rather  that  the  expectations  characterize  the  primary  interests  of  each  category.  That  is, 
the  service  members’  expectations  center  around  outcomes  that  lead  to  similar  dispositions  and  their 
right  to  due  process,  and  the  military  departments’  expectations  center  around  expeditious  processing 
and  efficient  operations  of  the  system. 

^Performance  measures  that  focus  on  fair  treatment  of  the  service  member  could  supplement  the  surveys, 
and  perhaps  highlight  where  perceptions  differ  from  the  facts.  Personal  perceptions,  in  the  end,  however, 
are  likely  to  be  the  primary  determinants  of  whether  the  member  believes  his  or  her  expectations  are 
fulfilled  or  not. 

^In  addition,  we  recommend  that  the  DoD  Inspector  General  periodically  assess  how  closely  the  military 
departments  are  following  the  directions  in  the  DoD  Instruction.  For  example,  the  DoD  Instruction 
currently  directs  the  PEB  to  include  its  rationale  for  the  fitness  determination  in  its  report.  However, 
primary  participants  reported  that  this  direction  was  not  being  followed  and,  as  a  result,  disability  policy 
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Relying  on  the  perceptions  of  customers  has  certain  advantages.  In  fact,  customer 
perceptions  constitute  an  important  component  of  the  management  information 
system  this  chapter  describes.  Reliance  on  customer  perceptions  falls  short,  however, 
in  terms  of  providing  leadership  with  the  full  range  of  information  it  needs  to  carry 
out  its  assigned  responsibilities.  Therefore,  we  recommend  a  set  of  complementary 
measures,  although  as  proxies  only,  that  provide  a  more  objective  and  quantitative 
indication  of  how  well  the  DES  fulfills  customer  expectations. 

These  complementary  measures  comprise  two  types:  outcome  measures  and  output 
measures,  as  defined  earlier  in  this  chapter.  Outcome  measures  provide  the  best  in¬ 
sight  into  the  operation  of  the  system  as  a  whole,  but  they  are  less  helpful  in  deter¬ 
mining  the  effect  of  interventions  on  the  performance  of  the  DES.  Outcome  measures 
have  their  genesis  in  the  four  customer  expectations  listed  in  Table  6.1.  Output 
measures,  on  the  other  hand,  provide  insight  into  the  operation  of  major  parts  of  the 
DES.  Output  measures  lack  the  comprehensiveness  of  outcome  measures  but  pro¬ 
vide  significantly  more  insight  into  the  effect  of  interventions  on  system  perfor¬ 
mance. 

Both  types  of  measures,  however,  directly  or  indirectly  reflect  how  well  the  DES  ful¬ 
fills  the  performance  expectations  of  external  customers.  The  remainder  of  this  sec¬ 
tion  discusses  outcome  measures  (because  they  are  most  closely  related  to  customer 
expectations):  the  next  section  of  this  chapter  discusses  output  measures  in  more 
depth. 

Two  outcome  measures  of  system  performance— case  variability  and  number  of 
appeals— capture  critical  aspects  of  service  member  expectations  (that  is,  similar  dis¬ 
positions  and  due  process).  Case  variability  is  an  indicator  of  consistency  in  the  sys¬ 
tem.  If  the  system  treats  similar  service  members  with  similar  disabilities  in  similar 
ways,  we  consider  the  system  to  be  consistent  in  its  operation.®  The  number  of  ap¬ 
peals  is,  admittedly,  a  very  gross  proxy  for  due  process.  Appeals,  however,  can  signal 
a  lack  of  confidence  in  the  system  (service  members  questioning  every  determina¬ 
tion),  or  alternatively,  that  the  system  is  operating  as  intended  (by  affording  service 
members  the  opportunity  to  exercise  their  rights).  Consequently,  this  measure  is 
preliminary  and  potentially  suggestive  of  problem  areas,  not  a  hard  indicator  of  how 
well  the  system  is  performing  with  regard  to  due  process.  We  recommend  a  combi¬ 
nation  of  metrics,^  which  are  discussed  later,  to  provide  greater  insight  into  this  as¬ 
pect  of  service  members’  expectations.® 


was  being  inconsistently  applied.  In  this  particular  situation,  inappropriate  considerations  (such  as  those 
based  on  personal  bias)  would  be  more  difficult  to  employ  if  the  direction  were  followed. 

^We  recommend  that  the  DoD  Inspector  General  periodically  sample  completed  cases  and  assess  the 
similarity  of  final  dispositions  (fitness,  ratings,  personnel  action)  for  similarly  situated  service  members 
(within  and  across  military  departments).  These  surveys  differ  from  the  "customer  satisfaction"  surveys 
that  focus  on  service  member  perceptions  of  whether  the  system  fulfilled  the  member's  expectations  (and, 
as  noted  earlier,  could  provide  a  basis  for  comparing  perceptions  with  the  facts). 

The  report  uses  the  term  metrics  to  mean  the  specific  data  to  collect,  organize,  and  store  in  the 
management  information  system.  Several  metrics  may  be  associated  with  a  specific  outcome  or  output 
measure.  ^ 

g 

Based  on  our  review  of  the  system,  we  found  no  indication  that  service  members  were  being  denied  due 
process.  This  appears  to  be  a  major  example  of  a  part  of  the  DES  that  Is  working  effectively.  However,  be- 
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Two  other  outcome  measures  of  system  performance— time  to  replace  “broken”  ser¬ 
vice  members^  and  total  cost  of  the  system}^^ — capture  critical  aspects  of  service  ex¬ 
pectations  (that  is,  expeditious  processing  and  efficient  operations).  Time  to  replace 
a  service  member  is  an  important  measure  from  the  perspective  of  unit  comman¬ 
ders.  The  less  time  between  the  onset  of  an  injury  or  disease  that  adversely  affects  a 
service  member’s  unit  performance  and  the  arrival  of  a  replacement,  the  better  the 
DES  meets  the  unit  commander’s  expectation.  The  lower  the  total  cost  of  the  system 
(the  denominator  in  an  assessment  of  efficiency),  the  better  the  system  meets  the 
service’s  expectation. 

Figure  6.1  portrays  the  relationship  between  the  proposed  purpose  of  the  DES  and 
customer  perspectives  and  expectations,  and  the  relationships  between  customer 
expectations  and  the  suggested  outcome  measures,  as  discussed  in  this  section. 
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Figure  6.1 — Proposed  Purpose  of  the  Disability  Evaluation  System  Linked  to  Suggested 

Outcome  Measures 


cause  due  process  is  a  critical  element  of  overall  system  performance,  it  is  an  aspect  that  requires  constant 
vigilance.  This  is  one  reason  that  we  believe  it  should  have  a  central  role  in  the  purpose  statement  and  in 
the  measures  used  to  assess  system  performance. 

^This  term  refers  to  service  members  who  are  under  a  physician’s  care  and  unable  to  perform  their  normal 
duties,  and  service  members  who  are  deemed  unfit  by  the  PEB  to  perform  the  duties  of  their  office,  grade, 
rank,  or  rating. 

l^Total  cost  of  the  system  includes  the  variable  costs  of  operating  the  DES  plus  investments  in  training 
and  automation.  Although  not  visible  in  the  budget  process,  investments  in  training  and  automation 
should  be  amortized  and  future  benefits  included  in  the  cost  perspective. 
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Unfortunately,  by  their  nature,  outcome  measures  (and,  direct  measures  of  customer 
perceptions)  are  lagging  indicators  of  system  performance.  Monitors  cannot  deter¬ 
mine  how  well  a  program  or  system  is  performing  until  after  the  fact.  Although  they 
are  useful  in  hindsight  and  in  the  final  determination  of  performance,  outcome  mea¬ 
sures  are  less-than-desirable  measures  for  ensuring  desired  results. 

Identifying  and  acting  on  leading  indicators  are  key  actions  to  ensure  desired  system 
results  as  required  by  the  responsibilities  assigned  in  DoDI  1332.38.  Consequently, 
we  propose  an  expanded  set  of  performance  measures  that  encompass  leading 
indicators  by  measuring  system  outputs.  Output  measures  reflect  intermediate 
performance  results  of  key  parts  of  tbe  system  that  contribute  to  system  outcomes. 
Output  measures  also  reflect  a  mix  of  lagging  and  leading  indicators,  where  each 
indicator  is  linked  to  the  other  indicators  in  a  logical  way.  The  following  describes  an 
integrated  set  of  performance  measures  for  assessing  how  well  the  DES  accomplishes 
its  purpose.  Performance  measures  include  both  outcome  and  output  measures. 

DEVELOPING  AN  INTEGRATED  SET  OF  PERFORMANCE  MEASURES 

To  guide  our  search  for  leading  indicators  of  system  performance,  we  asked  two  fun¬ 
damental  questions:  (1)  What  moves  lagging  indicators  in  the  right  direction?  and 
(2)  What  can  those  charged  with  overseeing  the  system  control  and  be  held  account¬ 
able  for  that  will  lead  to  the  desired  outcomes?  To  answer  these  questions,  we  identi¬ 
fied  output  measures  that,  on  the  one  hand,  interventions  can  influence  and,  on  the 
other,  can  lead  to  changes,  directly  or  indirectly,  in  the  outcome  measures. 

Two  types  of  interventions  significantly  affect  output  measures:  process  improve¬ 
ments  and  enhanced  primary  participant  competence.^  ^  Process  improvements  lead 
to  faster  cycle  times  that  affect  how  quickly  a  case  is  resolved  and  impact  how  well 
the  DES  fulfills  both  service  member  and  service  expectations.  Enhanced  primary 
participant  competence  also  supports  faster  cycle  times  by  increasing  productivity. 
In  addition,  it  leads  to  reduced  operating  costs  and  better-educated  commanders 
and  service  members  who  then  become  more-effective  participants  in  the  DES. 

Figure  6.2  portrays  the  multiplicity  of  relationships  among  interventions,  output 
measures,  and  outcome  measures. 


In  other  words,  interventions  in  the  form  of  process  improvements  and  enhanced  staff  competency  are 
actions  that  lead  to  specific  changes  in  system  performance  that  affects  how  well  the  DES  is  fulfilling  both 
military  service  and  service  member  expectations.  To  put  it  another  way,  the  goal  of  a  strategy  for 
improving  the  operation  of  the  DES  is  to  better  achieve  the  purpose  of  the  system;  two  means  for 
achieving  that  end  are  process  improvements  and  enhanced  staff  competency. 
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Figure  6.2 — Linking  Interventions  to  Outcome  Measures  Through  Output  Measures 


Process  Improvements  Intervention 

The  OSD  can  assess  the  impact  of  the  majority  of  process  improvements  by  focusing 
on  three  output  measures;  processing  time,  number  of  reworks,  and  time  to  promul¬ 
gate  policy  change.  Processing  time  is  the  length  of  time  it  takes  a  service  member 
represented  by  a  medical  board,  to  proceed  through  the  various  phases  of  the  DBS. 
Processing  time  focuses  more  narrowly  than  the  outcome  measure  time  to  replace,  in 
that  it  focuses  attention  on  what  parts  of  the  DBS  contribute  most  to  the  overall  time- 
to-replace  outcome  measure.  Although  the  number  of  reworks  (for  example,  in¬ 
stances  in  which  medical  boards  are  referred  back  to  an  earlier  phase  in  the  process 
for  lack  of  information)  affects  the  processing  time,  it  is  a  major  source  of  delay  and, 
therefore,  deserves  specific  attention.  The  time  a  military  department  takes  to  pro¬ 
mulgate  policy  change  or  guidance  also  affects  the  consistency  of  policy  application, 
particularly  across  military  departments. 

Enhanced  Primary  Participant  Competency  Intervention 

The  OSD  can  assess  the  impact  of  the  majority  of  actions  directed  at  enhanced  pri¬ 
mary  participant  competency  using  three  output  measures:  (1)  productivity  (medical 
boards  processed  per  capita),  (2)  processing  cost  per  medical  board,  and  (3) 


l^One  military  department  has  not  yet  published  a  formal  regulation  incorporating  changes  in  DoD 
Instruction  1332.38  published  in  1996,  though  it  has  distributed  memoranda  that  broadcast  the  changes. 
Although  the  operation  of  the  military  department  DBS  conforms  to  the  guidance  contained  in  the  DoD 
Instruction,  primary  participants  and  other  interested  parties  rely  on  the  outdated  (circa  1990) 
implementing  regulation  for  reference,  which  could  be  the  source  of  incorrect  information  leading  to 
variability  in  processing  medical  boards  and  implementing  DoD  disability  policy. 
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percentage  of  primary  participants  certified,  augmented  with  one  input  measure — 
total  resources  devoted  to  the  DES. 

Productivity  is  an  important  measurement  of  the  contribution  of  different  primary 
participants  to  system  performance  including  PEBLOs,  members  of  the  Informal  PEB 
and  Formal  PEB,  PEB  administrative  action  officers,  and  others. 

Productivity  and  total  resources  affect  the  cost  per  medical  board;  however,  the  cost- 
per-medical-board  measure  provides  greater  insight  into  changes  to  the  total  cost  of 
the  system  than  do  the  other  measures. 

Certification  is  a  key  indicator  of  primary  participant  competency  and  one  of  the 
most  useful  of  the  output  measures. 

Total  resources  devoted  to  the  DES  include  manpower  numbers  and  cost,  informa¬ 
tion  support,  training,  and  operation  and  maintenance  funds  (other  than  for  civilian 
salaries  and  training).  A  measure  of  total  resources  focuses  attention  on  the  funda¬ 
mental  level  of  input  into  the  DES. 


A  Framework  for  Integrating  Outcome  and  Output  Performance  Measures 

Arguably,  the  output  measures  associated  with  process  improvements  and  enhanced 
primary  participant  competency  are  leading  indicators  of  system  performance.  For 
example,  as  productivity  increases,  processing  time  and  the  time  to  replace  a  service 
member  found  unfit  decreases.  It  may  take  a  while,  however,  for  outcome  measures 
to  reflect  productivity  increases.  Similarly,  as  the  percentage  of  primary  participants 
who  are  certified  in  their  particular  body  of  DES  knowledge  and  skills  that  are  re¬ 
quired  to  produce  desired  on-the-job  results  increases,  service  members  receive  bet¬ 
ter  information,  make  more-informed  appeals,  and  more-positively  assess  how  well 
the  DES  fulfilled  their  expectations,  in  addition  to  other  benefits. 

We  constructed  a  framework  of  hypothetical  relationships  among  the  system  per¬ 
formance  measures,  which  is  shown  in  Figure  6.3.  A  similar  or  opposite  impact  that  a 
change  in  one  performance  measure  has  on  another  performance  measure  is  de¬ 
noted  in  the  figure  with  a  plus  or  minus  sign.  For  example,  the  plus  sign  next  to  the 
arrow  pointing  upward  from  “Total  Resources”  in  the  figure  suggests  that  as  total  re¬ 
sources  increase,  the  percentage  certified  also  increases  (that  is,  the  measure  moves 
in  the  same  direction).  Conversely,  the  minus  sign  next  to  the  arrow  pointing  to  the 
left  of  Total  Resources  suggests  that  as  total  resources  increase,  the  time  to  promul¬ 
gate  policy  change  does  just  the  opposite;  that  is,  it  decreases.  Consequently,  as  the 
time  to  promulgate  policy  change  decreases,  case  variability  also  decreases  (that  is,  it 
moves  in  the  same  direction,  as  noted  by  the  plus  sign). 


13 

The  objective  of  a  certification  process  is  to  produce  a  fully  capable  human  resource.  Certification  might 
consist  solely  of  evidence  of  training,  or  evidence  of  training  plus  some  amount  of  experience;  it  might  also 
entail  evidence  of  periodic  continuing  education.  There  might  also  be  a  different  certification  process  for 
each  primary  participant  population  (for  example,  for  physicians  being  certified  in  writing  narrative 
summaries  and  PEBLOs  being  certified  in  their  much  broader  body  of  DES  knowledge). 
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Figure  6.3— Effect  of  a  Change  in  One  Performance  Measure  on  Another 
Performance  Measure 


As  noted  earlier  in  this  chapter,  the  number  of  appeals  is  a  very  gross  proxy  for  due 
process,  an  outcome  measure  related  directly  to  service  member  expectations.  We 
indicate  the  ambiguity  of  this  measure  with  a  plus/minus  sign  next  to  “Number  of 
Appeals”  in  Figure  6.3.^^ 

Figure  6.3  highlights  the  central  role  that  certification  plays  in  the  overall  perfor¬ 
mance  measurement  process.  It  influences,  directly  or  indirectly,  nearly  every  other 
performance  measure.  Only  the  level  of  total  resources,  an  input  into  the  system,  is 
more  influential.  The  primacy  of  certification  in  relation  to  the  other  measures  of 
system  performance  is  congruent  with  the  emphasis  we  place  on  the  training  inter¬ 
vention  in  Chapter  5.  Figure  6.3  also  suggests  that  a  decrease  in  resource  allocations 
would  produce  results  that  are  different  from  those  we  just  described.  The  figure  also 
identifies  where  to  focus  attention  to  measure  system  performance  in  order  to 
determine  if  the  level  of  resources  produces  the  intended  result. 


1‘^An  increase  in  the  percentage  certified  could  lead  to  an  increase  in  the  number  of  appeals  (for  example, 
as  a  result  of  increased  service  member  awareness)  or  to  a  decrease  in  the  number  of  appeals  (for  example, 
as  a  result  of  acquiring  more  information  regarding  the  conditions  under  which  an  appeal  would  be 
granted). 
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Based  on  our  review  of  the  DES  and  our  conversations  with  many  of  its  primary  par¬ 
ticipants,  these  performance  measure  relationships  have  face  validity.  As  noted  ear¬ 
lier,  the  OSD  should  treat  the  relationships  illustrated  in  the  figure  as  hypothetical 
ones.  The  validity  of  these  interrelationships  will  emerge  over  time;  if  they  do  not 
prove  to  be  accurate,  or  if  other  aspects  of  the  DES  increase  in  importance,  the  OSD 
should  revise  the  measures  and  relationships  portrayed  in  Figure  6.3.  Revising  per¬ 
formance  measures  and  the  relationships  among  measures  as  needed  is  a  critical 
part  of  monitoring  the  system  over  time,  in  accordance  with  the  responsibilities  as¬ 
signed  by  DoD  Instruction  1332.38. 

Focusing  solely  on  outcome  measures  puts  the  DES  leadership  in  the  position  of 
reacting  to  unfavorable  results  only  after  the  fact.  In  this  case,  leadership  lacks  in¬ 
formation  to  assess  whether  the  interventions  that  were  employed  to  maintain  or 
improve  certain  results  were  inadequately  deployed  or  were  the  wrong  interventions 
altogether.  The  framework  illustrated  in  this  section  is  a  strategic  performance  man¬ 
agement  tool  for  carrying  out  the  responsibilities  assigned  by  DoD  Instruction 
1332.38  and  for  identifying  areas  that  deserve  increased  attention  and  focus, ^5  with 
the  intent  of  better  achieving  the  purpose  of  the  Disability  Evaluation  System,  and 
not  acting  as  a  report  card.*® 

IMPLEMENTING  A  MEASUREMENT  SYSTEM 

Most  of  the  performance  measures  described  in  this  chapter  have  multiple  dimen¬ 
sions.  To  capture  the  richness  underlying  these  measures,  and  thereby  create  a 
complete  picture  of  system  performance  to  identify  potential  problems  quickly 
enough  to  take  corrective  action  and  assess  the  impact  of  the  interventions,  we  rec¬ 
ommend  a  number  of  metrics  for  each  performance  measure.  These  metrics  are 
summarized  in  tabular  format  at  the  end  of  this  chapter;  a  detailed  narrative  de¬ 
scription  appears  in  Appendix  H. 

General  Observations 

We  begin  with  some  general  observations  regarding  the  metrics. 

First  observation:  A  performance  measurement  system  requires  a  sizable  collective 
effort  and  a  significant  amount  of  resources.  For  a  performance  measurement 
system  to  be  effective,  OSD  and  military  department  leadership  must  commit  to 
providing  the  resources— monetary,  work  force,  and  collection  method  resources— 
needed  to  obtain  the  data  with  which  to  compute  the  metrics  used  to  monitor  the 


*®To  be  most  effective,  a  review  of  performance  measures  such  as  that  outlined  in  this  section  should 
involve  senior  civilian  and  military  officials  from  the  military  departments,  and  not  just  the  primary 
participant  of  the  DES.  To  carry  out  its  responsibilities,  senior  leadership  should  use  the  framework 
suggested  here  and  modify  that  framework  if  it  is  found  to  be  ineffective  in  explaining  the  effects  of  major 
interventions.  ' 
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It  is  important  to  note  that  this  framework  is  not  the  means  for  evaluating  individual  primary 
participants  or  even  a  military  department’s  DES.  Rather,  it  is  a  device  for  focusing  attention  on  and 
monitoring  the  critical  performance  measures  and  holding  the  appropriate  primary  participants  or 
military  department  accountable  for  change  in  the  right  direction  for  each  performance  measure. 
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system.  Ideally,  the  administrators  of  the  DBS  (PEBLOs,  disability  evaluation 
counselors,  patient  administrators,  and  PEB  action  officers)  enter  the  data  into  an 
automated  system  designed  to  monitor  performance.  Many  of  the  metrics 
recommended  in  this  report  are  suited  for  this  method  of  collection.  Designing  an 
automated  management  information  system  requires  early  decisions  on  the  form 
and  content  of  the  data  the  military  departments  will  collect  and  provide  to  the  OSD. 

In  the  absence  of  an  automated  system,  to  ease  the  burden  on  administrative  per¬ 
sonnel  and  increase  the  availability  and  accuracy  of  the  data  collected,  we  recom¬ 
mend  that  the  ASD/FMP  direct  the  Director  of  Officer  and  Enlisted  Personnel 
Management  to  consult  with  a  small  group  of  experienced  DES  experts  representing 
the  military  departments’  PEBs  and  the  Office  of  the  Surgeons  General  to  decide  on 
the  design  of  a  common  hard  copy  form  for  gathering  and  recording  the  data. 
Because  the  medical  board  is  the  central  administrative  element  around  which  the 
DES  operates,  we  envision  attaching  this  form  to  the  front  of  the  medical  board.  The 
form  will  contain  data  already  part  of,  or  that  can  be  derived  from,  the  medical 
board;  regardless,  the  intent  is  to  consolidate  the  collected  data  in  an  easily  accessi¬ 
ble  location,  such  as  the  front  of  the  medical  board. 

Second  observation:  To  motivate  DES  performance  improvement  in  the  military  de¬ 
partments,  we  recommend  that  the  OSD  establish  benchmarks  for  key  metrics  based 
on  trends,  a  DoD  standard,  and/or  DoD  averages.  Most  of  the  metrics  proposed  in 
the  following  section  are  based  on  historical  data.  Consequently,  for  most  of  the 
metrics,  the  appropriate  benchmark  is  the  trend  that  the  metric  exhibits  over  time 
(that  is,  a  self-improvement  benchmark)  or  how  the  metric  compares  with  a  DoD 
standard  or  to  the  average  value  of  the  metric  across  the  DoD  as  a  whole  (that  is,  a 
comparative  benchmark).  Although  a  trend  does  not  constitute  a  firm  benchmark  to 
strive  for,  it  can  highlight  a  change  in  the  direction  of  DES  performance  and  can  be 
the  basis  for  continued  improvement.  Significant  changes  in  a  trend  or  significant 
deviation  from  the  DoD  standard  or  average  should  initiate  investigation  into  the 
cause  of  the  changes. 

Importantly,  the  framework  portrayed  in  Figure  6.3  lends  itself  to  developing  and 
specifying  OSD  performance  objectives  for  succeeding  time  periods  to  achieve  the 
set  of  desired  outcomes.  For  explicit  OSD  performance  objectives,  the  OSD  should 
require  the  military  departments  to  present  a  plan  for  achieving  the  OSD  objectives 
over  a  specified  time  period.  This  plan  should  delineate  the  specific  interventions  to 
be  made  and  the  military  department  performance  objectives  to  be  achieved  for  each 
performance  measure  during  that  period.  The  collection  of  metrics  reported  in  each 
period  describes  the  actual  performance  of  the  military  department  over  time;  the 
military  department  plan  sets  the  desired  performance  for  the  future. 

On  the  one  hand,  an  explicit  plan  is  a  means  by  which  the  OSD  holds  the  Secretaries 
of  the  military  departments  accountable  for  achieving  the  DBS’s  stated  purpose.  On 
the  other  hand,  it  is  the  means  by  which  the  officials  responsible  for  the  DES  in  each 


^^Processing  times  are  an  example  of  current  targets  set  by  the  OSD. 
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military  department  can  articulate  a  case  for  the  resources  needed  to  achieve  the  de¬ 
sired  outcomes. 

Third  observation:  Based  on  our  conversations  with  a  wide  range  of  primary 
participants,  we  concluded  that  a  major  source  of  variability  in  the  system  arises 
from  the  lack  of  information  within  the  system  about  how  the  system  is  operating. 
We  strongly  recommend  summarizing  the  performance  data  gathered  and  analyzed 
and  feeding  it  back  to  the  primary  participants  of  the  DES.  Regularly  reporting  DES 
performance  results  to  senior  officials  in  the  military  departments  and  the  OSD  in  a 
valid  and  meaningful  way  is,  also,  essential  to  enable  them  to  carry  out  their 
responsibilities. 

To  be  most  effective,  performance  monitoring  must  be  results-oriented  (and  not  fo¬ 
cused  on  inputs  or  processes),  built  into  the  operational  routines  of  the  system,  and 
reported  frequently  and  publicly.  However,  in  and  of  itself,  monitoring  performance 
is  a  waste  of  time  if  the  information  received  is  not  evaluated  and  acted  upon.  The 
outcomes,  performance  measures,  and  metrics  recommended  in  this  chapter  pro¬ 
vide  the  foundation  for  evaluating  and  acting. 


A  Set  of  Metrics 

We  recommend  that  the  OSD  direct  the  military  departments  to  collect,  analyze,  and 
submit  the  metrics  outlined  in  Tables  6.2  through  6.5.  Each  performance  measure- 
outcome,  output,  and  input— encompasses  several  metrics.  The  metrics  listed  in 
Tables  6.2  and  6.3  focus  on  the  outcome  measures  directly  related  to  the  assessment 
of  how  well  the  DES  fulfills  military  service  and  service  member  expectations.  The 
metrics  listed  in  Tables  6.4  and  6.5  focus  on  the  output  measures  related  to  the 
two  interventions,  process  improvements  and  enhanced  primary  participant 
competency. 

The  following  tables  briefly  summarize  each  metric,  the  metric’s  reporting  fre¬ 
quency,  the  data  source,  comparative  benchmark,  and  the  mechanism  for  gathering 
the  data.  Appendix  H  contains  a  more  detailed  description  of  these  metrics. 

Table  6.2 

Performance  Metrics  for  Assessing  Service  Member  Expectations 


Metric 


Distribution  of  medical 
boards  by  diagnostic 
category 

Statistical  analysis  of 
dispositions  (fitness, 
rating,  personnel  action) 
for  major  diagnostic 
categories 


Frequency _ Data  Source  Benchmark 

_ Case  Variability 

Quarterly  Medical  Boards  Trends 

forwarded  to 
Informal  PEB  (IPEB) 
in  previous  quarter 

Annual  Random  sample  of  Trends,  DoD 

PEB  decisions  standards 

rendered  from  the 
top  five  diagnostic 
categories  in  the 
previous  year 


Data-Gathering 

Mechanism 


Data  collection 
sheet  attached  to 
medical  board 

Sample  survey 
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Table  6.2 — Continued 


Metric 

Frequency 

Data  Source 

Benchmark 

Data  Gathering 
Mechanism 

Statistical  analysis  of 

Annual 

Random  sample  of 

Trends,  DoD 

Sample  survey 

dispositions  (fitness, 

PEB  decisions 

standards 

rating,  personnel  action) 

rendered  from  the 

for  special  diagnostic 

special  diagnostic 

categories  (e.g.,  HIV) 

categories  in  the 

previous  year 

Number  of  Appeals 

Percent  of  IPEB  decisions 

Quarterly 

IPEB  decisions 

Trends,  DoD 

Data  collection 

appealed  for  fitness 

rendered  in  the 

average 

sheet  attached  to 

previous  quarter 

medical  board 

Percent  of  IPEB  decisions 

Quarterly 

IPEB  decisions 

Trends,  DoD 

Data  collection 

appealed  for  fitness 

rendered  in  the 

average 

sheet  attached  to 

overturned 

previous  quarter 

medical  board 

Percent  of  IPEB  decisions 

Quarterly 

IPEB  decisions 

Trends,  DoD 

Data  collection 

appealed  for  rating 

rendered  in  the 

average 

sheet  attached  to 

previous  quarter 

medical  board 

Percent  of  IPEB  decisions 

Quarterly 

IPEB  decisions 

Trends,  DoD 

Data  collection 

appealed  for  rating 

rendered  in  the 

average 

sheet  attached  to 

overturned 

previous  quarter 

medical  board 

Percent  of  Formal  PEB 

Quarterly 

FPEB  decisions 

Trends,  DoD 

Data  collection 

(FPEB)  decisions 

rendered  in  the 

average 

sheet  attached  to 

appealed  for  fitness 

previous  quarter 

medical  board 

Percent  of  FPEB 

Quarterly 

FPEB  decisions 

Trends,  DoD 

Data  collection 

decisions  appealed  for 

rendered  in  the 

average 

sheet  attached  to 

fitness  overturned 

previous  quarter 

medical  board 

Percent  of  FPEB 

Quarterly 

FPEB  decisions 

Trends,  DoD 

Data  collection 

decisions  appealed  for 

rendered  in  the 

average 

sheet  attached  to 

rating 

previous  quarter 

medical  board 

Percent  of  FPEB 

Quarterly 

FPEB  decisions 

Trends,  DoD 

Data  collection 

decisions  appealed  for 

rendered  in  the 

average 

sheet  attached  to 

ratine  overturned 

previous  quarter 

medical  board 

General 

Percent  of  service 

Annual 

PEB  and  appellate 

Trends,  DoD 

100-percent  paper 

members  satisfied  with 

review  board 

standards 

survey 

disposition  decision 

decisions  rendered 

in  previous  year 

Percent  of  service 

Annual 

PEB  and  appellate 

Trends,  DoD 

100-percent  paper 

members  satisfied  with 

review  board 

standards 

survey 

process  (timeliness. 

decisions  rendered 

courtesy, 

in  previous  year 

responsiveness, 

assistance) 

Percent  of  service 

Annual 

FPEB  decisions 

Trends,  DoD 

100-percent  paper 

members  satisfied  they 

rendered  in 

standards 

survey 

received  due  process 

previous  year 

Number  of 

Annual 

Correspondence 

Trends,  DoD 

Review  of 

congressionals 

control 

average 

correspondence 

records 

General  Accounting 

When 

Report 

Internal  to 

Review  of  report 

Office  reports 

issued 

report 

Inspector  General  reports 

When 

Report 

Internal  to 

Review  of  report 

issued 

report 
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Table  6.3 


Performance  Metrics  for  Assessing  Military  Service  Expectations 


Metric 

Frequency 

Data  Source 

Benchmark 

Mechanism 

Total  System  Cost 

Total  resources  for  the 
operation  of  the  DES 

Annual 

Budget  data 

Trends 

Budget  analysis 

Pay  and  allowances  for 
service  members  not 

Annual 

Budget  data 

Trends 

Budget  analysis 

performing  duty 

The  cost  of  disability 
severance  pay 

Annual 

Budget  data 

Trends 

Budget  analysis 

Time  to  Replace 

For  service  members 
returned  to  duty, 
average  total  time  from 
referral  to  an  MTF  to 
return  to  duty,  broken 
out  by  diagnostic 
category 

Quarterly 

Medical  boards  of 
service  members 
returned  to  duty  in 
the  previous 
quarter 

Trends 

Data  collection 
sheet  attached  to 
medical  board 

For  service  members 
separated  or  retired, 
average  total  time  from 
referral  to  an  MTF  to 
termination,  broken 

Quarterly 

Medical  boards  of 
service  members 
terminated  in  the 
previous  quarter 

Trends 

Data  collection 
sheet  attached  to 
medical  board 

out  by  diagnostic 
category 

Average  total  time  on  the 
TDRL,  broken  out  by 
diagnostic  category 

Quarterly 

Medical  boards  of 
service  members 
removed  from  the 
TDRL  in  the 

Trends  in 
average 
time 

Data  collection 
sheet  attached  to 
medical  board 

Average  total  time  on 
limited  duty,  broken 
out  by  time  before 
referral  to  the  DES  and 
after  referral  and  by 
diagnostic  category 

Quarterly 

previous  quarter 
Medical  boards  of 
service  members 
referred  to  the  DES 
and  removed  from 
limited  duty  in  the 
previous  quarter 

Trends  in 
average 
time 

Data  collection 
sheet  attached  to 
medical  board 

Table  6.4 

Performance  Metrics  for  Assessing  Process  Improvements 


Metric 


Frequency  Data  Source 


Distribution  of  waiting  times  Quarterly 
for  narrative  summary 
dictation  (referral  to  MTF 
to  medical  board  dictated 
or  service  member 
returned  to  duty)  by 
diagnostic  categoiy^ 

Distribution  of  waiting  times  Quarterly 

for  IPEB  consideration 
(narrative  summary 
dictated  to  IPEB 
recommendation)  by 
diagnostic  category 


Processing  Time _ 

Medical  boards 
forwarded  to  IPEB  in 
previous  quarter 


IPEB  decisions 
rendered  in  previous 
quarter 


Benchmark  Mechanism 


Trends,  DoD  Data  collection 
standards  sheet  attached 
to  medical 
board 


Trends,  DoD  Data  collection 
standards  sheet  attached 
to  medical 
board 
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Table  6.4  (continued) 


Metric 

Frequency 

Data  Source 

Benchmark 

Mechanism 

Processing  Time 

Distribution  of  waiting  times 

Quarterly 

FPEB  decisions 

Trends,  DoD 

Data  collection 

for  FPEB  consideration 

rendered  in  the 

standards 

sheet  attached 

(IPEB  recommendation  to 

previous  quarter 

to  medical 

FPEB  decision)  by  diagnos- 

board 

tic  category 

Distribution  of  waiting  times 

Quarterly 

Post-PEB  appellate 

Trends,  DoD 

Data  collection 

for  completion  of  appellate 

review  board 

standards 

sheet  attached 

review  boards  (FPEB 

decisions  rendered 

to  medical 

decision  to  final  appellate 

in  the  previous 

board 

review  board  decision)  by 

quarter 

diagnostic  category 

Number  of  Reworks 

Percent  of  deficient 

Quarterly 

Medical  boards 

Trends 

Data  collection 

commanders’  letters  at 

received  by  the  IPEB 

sheet  attached 

IPEB,  by  reason 

in  the  previous 

to  medical 

quarter 

board 

Percent  of  deficient 

Quarterly 

Medical  boards 

Trends 

Data  collection 

commanders'  letters  at 

received  at  the  FPEB 

sheet  attached 

FPEB,  by  reason 

in  the  previous 

to  medical 

quarter 

board 

Percent  of  deficient  narrative 

Quarterly 

Medical  boards 

Trends 

Data  collection 

summaries  at  IPEB,  by 

received  at  the  IPEB 

sheet  attached 

reason 

in  the  previous 

to  medical 

quarter 

board 

Percent  of  deficient  narrative 

Quarterly 

Medical  boards 

Trends 

Data  collection 

summaries  at  FPEB,  by 

received  at  the  FPEB 

sheet  attached 

reason 

in  the  previous 

to  medical 

quarter 

board 

Percent  of  incomplete 

Quarterly 

Medical  boards 

Trends 

Data  collection 

medical  boards  at  IPEB,  by 

received  by  the  IPEB 

sheet  attached 

reason 

in  the  previous 

to  medical 

quarter 

board 

Percent  of  incomplete 

Quarterly 

Medical  boards 

Trends 

Data  collection 

medical  boards  at  FPEB, 

received  by  the 

sheet  attached 

by  reason 

FPEB  in  the  previous 

to  medical 

quarter 

board 

Time  to  transmit  information 

When 

Notification  of  policy 

Trends,  DoD 

Confirmation 

to  the  field 

issued 

implementation 

average 

memo  from 

military 

department 

Secretariat 

Time  to  update  military 

When 

Notification  of  policy 

Trends,  DoD 

Confirmation 

department  policy  state¬ 

issued 

implementation 

average 

memo  from 

ments 

military 

department 

Secretariat 

Time  to  update  training 

When 

Notification  of  policy 

Trends,  DoD 

Confirmation 

issued 

implementation 

average 

memo  from 

military 

department 

Secretariat 
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Table  6.5 

Performance  Metrics  for  Assessing  Enhanced  Primary  Participant  Competencies 


Medical  board  decisions 
rendered  per  IPEB 
member 


Medical  board  decisions 
rendered  per  FPEB 
member 


Medical  board  pro¬ 
cessing  completed  per 
full-time  PEBLO  and 
PEB  Administrative 
Action  Officer  assigned 
to  PEB 


Primary  participant 
satisfaction,  by 
primary  participant 
population 


Frequency _ Data  Source _ 

_ Productivity  (Boards  Per  Capita) 


Quarterly 


Quarterly 


Quarterly 


IPEB  decisions 
rendered  in 
previous  quarter; 
average  number  of 
IPEB  members 
FPEB  decisions 
rendered  in 
previous  quarter; 
average  number  of 
FPEB  members 
Total  medical  boards 
processed  in 
previous  quarter; 
average  number  of 
full-time  PEBLOs 
and  PEB  Adminis¬ 
trative  Action 
Officers  assigned  to 
PEB 

Survey  of  primary 
participants 


Benchmark 

r 

Trends,  DoD 
average 


Trends,  DoD 
average 


Trends,  DoD 
average 


Mechanism 

Data  collection 
sheet  attached  to 
medical  board 


Data  collection 
sheet  attached  to 
medical  board 


Data  collection 
sheet  attached  to 
medical  board 


Trends 


100-percent  paper 
survey 


Turnover,  by  primary 
participant  population 

Annual 

Percent  of  partici¬ 
pants  remaining  at 
end  of  year  who 
began  year 

Trends 

Manning 

documents 

Cost  Per  Medical  Board  Decision 

Total  system  cost  divided 
by  total  medical  board 
decisions  rendered 

Annual 

Budget  data;  medical 
board  decisions 
rendered  in  the 
previous  year 

Trends 

Budget  analysis; 
data  collection 
sheet  attached  to 
medical  board 

Percent  o 

f  Primary  Participants  Certified 

Percent  of  commanders' 
letters  submitted  by 
certified  commander 

Quarterly 

Commanders’ 
letters  forwarded  to 
IPEB  in  previous 
quarter 

Trends,  DoD 
standards 

Data  collection 
sheet  attached  to 
medical  board 

Percent  of  medical 
boards  dictated  by 
certified  physician 

Quarterly 

Medical  boards 
forwarded  to  IPEB 
in  previous  quarter 

Trends,  DoD 
standards 

Data  collection 
sheet  attached  to 
medical  board 

Percent  of  PEBLOs 
certified 

Quarterly 

Personnel  records 

Trends,  DoD 
standards 

Personnel  records 

Percent  of  IPEB 
members  certified 

Quarterly 

Personnel  records 

Trends,  DoD 
standards 

Personnel  records 

Percent  of  FPEB 
members  certified 

Quarterly 

Personnel  records 

Trends,  DoD 
standards 

Personnel  records 

Percent  of  PEB 
administrative  action 

Quarterly 

Personnel  records 

Trends,  DoD 
standards 

Personnel  records 

officers  certified 
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Table  6.5  (continued) 


Metric 

Frequency 

Data  Source 

Benchmark 

Mechanism 

Total  Resources 

Number  of  individuals, 
broken  out  by  DES 
phase  and  primary 
participant  population, 
devoted  to  the  DES 

Annual 

Budget  data 

Trends 

Budget  analysis 

Pay  and  allowances/ 
salaries  of  individuals 
broken  out  by  DES 
phase  and  primary 
participant  population, 
devoted  to  the  DES 

Annual 

Budget  data 

Trends 

Budget  analysis 

Information  manage¬ 
ment  system  costs 

Annual 

Budget  data 

Trends 

Budget  analysis 

Training  costs 

Annual 

Budget  data 

Trends 

Budget  analysis 

Operations  and 

Annual 

Budget  data 

Trends 

Budget  analysis 

maintenance  costs 
(other  than  training 
and  civilian  salaries) 


Chapter  Seven 

COSTS  AND  BENEFITS  OF  TRAINING 


As  discussed  in  Chapter  5,  we  recommend  that  the  Office  of  the  Assistant  Secretary  of 
Defense  develop  and  monitor  two  methods  of  knowledge-based  training — self- 
directed  computer-based  distance  training  and  classroom  training.  Although 
training  is  often  viewed  as  a  cost  of  doing  business,  it  can  be  more  appropriately 
characterized  as  an  investment  that  will  reap  benefits,  both  quantifiable  and 
nonquantifiable,  over  time. 

Also,  as  discussed  in  Chapter  6,  certification — a  measure  of  a  primary  participant’s 
capability  to  produce  the  desired  on-the-job  results — influences  a  number  of  the 
DBS  outcome  measures.  For  instance,  we  hypothesize  that  training  will  increase 
productivity  and  decrease  the  number  of  reworks,  thereby  decreasing  the  time  to 
replace  a  member  (a  quantifiable  benefit)  and  decreasing  case  variability  (a  non¬ 
quantifiable  benefit). 

Over  the  five-year  time  frame  of  the  cost/benefit  projection  shown  in  Table  7.1,  the 
quantitative  benefits  outweigh  the  costs  by  approximately  $2.39  million.  As  shown  in 
the  table,  during  the  development  year,  costs  outweigh  benefits,  assuming  Year  0  will 
be  dedicated  to  developing  the  training  packages.  During  Year  1  of  the  training,  costs 
again  outweigh  the  benefits.  However,  the  benefits  outweigh  the  costs  in  each  of  the 
remaining  years  (as  reduced  costs  reflect  the  impact  of  shorter  processing  times). 


Table  7.1 

Net  Present  Value  Quantifiable  Costs  and  Benefits 


Year 

NPV  Costs 

NPV  Benefits 

NPV  Total 

0 

$656,419 

_ 

($656,419) 

1 

$2,997,861 

$2,494,485 

($503,376) 

2 

$2,518,245 

$3,497,192 

$  978,947 

3 

$2,353,500 

$3,268,404 

$914,904 

4 

$2,199,532 

$3,054,583 

$855,050 

5 

$2,055,638 

$2,854,750 

$799,113 

Total 

$12,781,195 

$15,169,414 

$2,388,219 
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In  addition  to  the  $2.39  million  in  quantifiable  benefits  that  training  produces,  it  also 
produces  the  nonquantifiable  benefits  ^  of  consistent  policy  application,  increased 
job  and  customer  satisfaction,  and  increased  unit  readiness. 

This  chapter  presents  our  analysis  of  the  costs  and  benefits  of  our  training  recom¬ 
mendations.  Although  ex  ante  cost-benefit  analyses^  provide  useful  information  that 
can  be  used  when  deciding  whether  to  undertake  a  project,  they  are  based  on  esti¬ 
mates  of  the  future  costs  and  benefits.  Whether  or  not  the  estimates  hold  in  the  fu¬ 
ture,  the  process  of  conducting  cost-benefit  analyses  has  high  value  in  and  of  itself, 
as  it  helps  leaders  think  in  depth  about  specific  projects  and  their  associated  results. 
We  begin  this  chapter  with  a  discussion  of  general  assumptions  underlying  this 
analysis.  We  then  present  a  detailed  discussion  of  the  costs  and  benefits  of  our 
training  recommendations  and  conclude  with  a  presentation  and  discussion  of  the 
results  of  the  analysis. 


GENERAL  ASSUMPTIONS 

Ex  ante  analyses  are  by  necessity  based  upon  a  number  of  assumptions  due  to  uncer¬ 
tainty  regarding  future  results.  We  use  the  following  assumptions  throughout  our 
entire  analysis.  Specific  assumptions  regarding  the  quantification  of  costs  and  bene¬ 
fits  are  discussed  in  the  cost  and  benefit  sections  that  follow  this  list  of  assumptions. 

•  A  real  discount  rate  of  7  percent  is  used  in  present-value  calculations  in  accor¬ 
dance  with  Office  of  Management  and  Budget  Circular  A-94  guidelines. 

•  All  costs  and  benefits  are  expressed  in  fiscal  year  2000  real  dollars. 

•  We  count  only  the  costs  and  benefits  of  the  military  departments  and  the  OSD  in 
this  analysis. 

•  We  assume  a  five-year  time  horizon  for  the  training.  We  chose  a  five-year  time 
frame  for  the  lifetime  of  the  training  intervention,  a  relatively  short  time  frame, 
for  two  reasons.  First,  because  of  the  rapid  changes  in  computer  technology  and 
current  trends  toward  infinite  bandwidth  with  negligible  costs,  it  is  likely  that  the 
OSD  will  choose  to  update  and  modernize  its  self-directed  computer-based  dis¬ 
tance  training  significantly  within  the  next  decade.  Second,  in  interventions  such 
as  this  one  that  have  up-front  costs  with  benefits  that  accrue  in  later  years, 
shorter  time  horizons  place  heavier  emphasis  on  the  costs  of  the  intervention 
than  on  its  benefits.  In  this  analysis,  we  select  a  shorter  time  horizon  in  order  to 
provide  a  conservative  estimate  of  the  impact  of  our  recommended  training. 

•  Fiscal  year  2000  regular  military  compensation  pay-grade  averages  are  used  to 
compute  average  daily  pay  figures  for  military  personnel,  both  primary  partici- 


Technically,  analysts  could  attempt  to  place  monetary  values  on  these  benefits  through  contingent 
valuation  techniques  whereby  willingness  to  pay  for  the  benefit  is  elicited  through  survey  techniques. 

An  ex  ante  cost-benefit  analysis  occurs  before  a  policy  or  program  is  in  place  and  can  assist  in  the  decision 
about  whether  resources  should  be  allocated  to  that  program. 
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pants  and  service  members  progressing  through  the  system.  Average  daily  pay 
figures  for  military  personnel  reflect  a  365-workday  year. 

•  Civilian  standard  composite  pay  rates  by  grade  are  used  to  compute  the  average 
daily  pay  figures  for  civilian  personnel.^  Average  daily  pay  figures  for  civilian  per¬ 
sonnel  reflect  a  260-day  work  year. 

•  We  did  not  assume  any  quantifiable  benefits  or  costs  from  increased  consistent 
application  of  disability  policy,  specifically  fitness  and  ratings  decisions,  within 
and  among  the  military  departments.  We  believe  benefits  will  accrue  from  in¬ 
creased  consistent  application  as  a  result  of  the  training  recommendations;  how¬ 
ever,  within  the  scope  of  the  study,  we  did  not  attempt  to  gather  any  data  regard¬ 
ing  the  degree  or  impact  of  inconsistent  fitness  and  rating  decisions. 

COSTS 

We  estimate  that  the  net  present  value  (NPV)  of  the  cost  of  the  training  recommen¬ 
dations  is  $12.8  million  over  six  years.  This  figure  includes  the  cost  of  self-directed 
computer-based  distance  training  and  classroom  training. 

To  estimate  the  cost  of  the  training  interventions,  we  made  assumptions  about 
training  development  and  delivery  details,  such  as  the  number  of  individuals  devel¬ 
oping  training  and  the  length  and  firequency  of  training  for  both  the  self-directed 
computer-based  distance  training  and  the  classroom  training.  Table  7.2  details  the 
assumptions  upon  which  the  cost  analysis  is  based. 

Cost  of  Self-Directed  Computer-Based  Distance  Training 

We  estimate  that  self-directed  computer-based  distance  training  will  cost  $10.55 
million  over  the  five-year  period.  Table  7.3  details  the  cost  of  this  training  for  the 
participants  from  each  military  department. 

The  vast  majority  (94  percent)  of  the  cost  of  self-directed  computer-based  distance 
training  is  the  opportunity  cost  of  the  participants  who  are  engaging  in  the  training. 
Appendix  I  provides  data  that  show  these  opportunity  costs  broken  down  by  military 
department  and  training  population. 

We  do  not  include  computer  costs  because  we  assume  that  every  trainee  will  have 
access  to  an  existing  computer."^  If  this  assumption  proves  to  be  false,  an  additional 
cost  will  accrue. 


^Fiscal  year  1999  data  were  inflated  by  3.8  percent  to  provide  fiscal  year  2000  estimates. 

■^We  based  this  assumption  on  information  we  received  during  interviews  with  DBS  primary  participants. 
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Table  7.2 

Training  Assumptions  for  the  Two  Methods  of  Knowledge-Based  Training 


Assumption  Self-Directed  Computer-Based  Classroom  Training 

_ _ _ Distance  Training 

Development  time^  Initial:  five  months  Initial;  one  mondi 

Yearly  updates:  one  week  Yearly  updates:  one  week 

Internal  developers  Three  subject  matter  experts  Three  subject  matter  experts 

External  developers  Web-based  training  developer  None 

Training  population  All  primary  participants  Fifty  participants  per  session — 

primarily  PEB  members  and 
approving  authorities  and 
post-PEB  Appellate  Review 
Board  members 

Frequency  One  time  per  participant  Presented  quarterly;  participants 

attend  as  needed 

Length  PEBLOs;  Four  days  Four  days 

PEB  members  and  approving 
authorities:  Four  days 
Appellate  Review  Board 
members:  Four  days 
Physicians:  Four  hours 
Commanders:  One  hour 
PEB  Administrative  Action 
Officers:  Two  days 
Patient  Administrators:  One  day 


Location  Trainee's  duty  station  OSD  or  a  militar>^  department’s 

facilities  in  the  Washington, 

_ _ _ D.C.,  metropolitan  area 

^The  development  time  estimated  here  is  based  in  the  fact  that  the  military  departments  have  subject 
matter  experts  with  experience  in  developing  training  curricula  and  that  basic  modules  will  be  used  in 
each  of  the  population-specific  training  packages.  If  development  time  exceeds  our  estimate,  training 
costs  will  increase. 


Table  7.3 

Cost  of  Self-Directed  Computer-Based  Distance  Training 


Subject  Department  Department  Department 

Matter  Outside  of  the  of  the  of  the 

Year  Experts  Developer _ Army _ Navy  Air  Force  Total  Cost  NPV  Cost 

0  $130,349  $500,000  —  _  $578,210  $630,349 

1  $6,517  $5,000  $1,003,760  $1,141,070  $425,138  $2,581,486  $2,412,604 

2  $6,517  $5,000  $880,935  $1,019,006  $356,252  $2,267,709  $1,980,705 

3  $6,517  $5,000  $880,935  $1,019,006  $356,252  $2,267,709  $1,851,126 

4  $6,517  $5,000  $880,935  $1,019,006  $356,252  $2,267,709  $1,730,025 

5  $6,517  $5,000  $880,935  $1,019,006  $356,252  $2,267,709  $1,616,845 

XQjal _ $162,937  $525,000  $4,527,499  $5.217,092  $1,850,145  $12,230,533  $10,55 1 ,673 


We  estimate  that  outside  development  costs  are  $500,000  in  Year  0.  This  cost  is  based 
on  the  assumption  that  (1)  the  outside  developer  will  provide  a  professional  training 
developer  to  work  with  the  subject  matter  experts  to  develop  the  training;  (2)  a  core 
training  module  will  be  used  in  all  of  the  training  packages;  and  (3)  five  population- 
specific  training  packages  will  be  developed. 
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The  format  of  self- directed  computer-based  distance  training  can  cover  a  wide  range 
of  options  and  prices,  from  video-  and  audio-intensive  sessions  (which  can  be  ex¬ 
pensive)  to  purely  text-based  sessions  (which  are  typically  less  expensive).  The  costs 
shown  in  Table  7.4  are  based  on  a  primarily  text  format,  as  it  is  unknown  whether  all 
of  the  primary  participants  vdll  have  computers  with  the  capacity  to  handle  the  video 
and  audio  formats.  We  estimate  that  it  will  cost  $5,000  to  update  the  self-directed 
computer-based  distance-training  packages.  Ideally,  these  updates  will  be  informed 
by  data  gathered  by  the  management  information  system  that  we  recommend  in 
Chapter  6. 

Cost  of  Classroom  Training 

We  estimate  that  five  years  of  classroom  training  will  cost  $2.23  million.  As  shown  in 
Table  7.4,  the  vast  majority  of  the  cost  of  classroom  training  comes  from  participant 
time  and  travel.  The  participant  time  represents  the  opportunity  cost^  of  the 
participants  attending  training,  a  cost®  to  the  military  departments.  When  calculating 
this  cost,  we  included  one  day  of  travel  time  as  well  as  the  four  days  of  training. 

We  assume  that  the  OSD  will  budget  for  all  other  costs — instructors,  travel,  and  ma¬ 
terials.  The  travel  cost  includes  plane  fare,^  hotel,  and  per  diem  costs  for  all  partici¬ 
pants  and  instructors.®  We  estimate  the  material  costs  of  classroom  training  at  $50 
per  participant  per  training  session.  As  noted  in  Table  7.2,  this  analysis  assumes  that 
the  training  occurs  at  a  military  facility  that  would  otherwise  have  been  unused.  If 
this  assumption  is  invalid  and  the  training  sessions  take  place  at  an  outside  facility  or 
the  facility  would  have  been  in  use  otherwise,  an  additional  cost  will  accrue. 

BENEFITS 

The  premise  underlying  this  chapter  is  that  training  does  produce  the  hypothesized 
benefits.  We  must  emphasize,  though,  that  training  will  produce  payoffs  only  if 
training  content  and  delivery  focus  on  precise  DES  topics  that  constitute  the  bodies 
of  knowledge  and  specific  skills  required  to  produce  desired  on-the-job  results.  In 
other  words,  poorly  developed  training  is  unlikely  to  produce  the  desired  benefits. 


^Opportunity  cost  represents  what  is  foregone  by  undertaking  a  given  action.  If  the  participants  were  not 
engaged  in  OSD-provided  training,  they  would  be  engaged  in  other  work  activities.  We  estimate  the 
opportunity  cost  of  participation  in  training  throughout  the  analysis  as  the  participant’s  pay  for  the 
designated  unit  of  time. 

®We  assumed  that  the  training  population  would  primarily  consist  of  PEB  members  and  approving 
authorities  and  post-PEB  Appellate  Review  Board  members.  As  a  result,  this  opportunity  cost  is  based 
upon  an  0-6’s  average  salary.  We  expect  that  others,  including  Medical  Evaluation  Board  members  and 
approving  authorities  and  attorneys,  may  also  attend  this  training.  Because  the  average  rank  of  these 
members  is  typically  lower  than  0-6,  our  estimate  may  overestimate  the  actual  opportunity  cost. 

^We  estimate  plane  fare  to  cost  $300  per  person. 

^We  expect  that  this  figure  may  overestimate  the  travel  cost  because  some  participants  or  instructors  may 
be  stationed  in  the  area  where  the  training  occurs. 
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Table  7.4 

Cost  of  Classroom  Training 


Year 

Instructor 

Participant 

Time 

Subject  Matter 
Travel  Experts 

Materials 

Total  Cost 

NPVCost 

0 

— 

— 

$26,070 

— 

$26,070 

$26,070 

1 

$17,580 

$293,000 

$210,304 

$6,517 

$10,000 

$537,401 

$502,244 

2 

$17,580 

$293,000 

$210,304 

$6,517 

$10,000 

$537,401 

$469,387 

3 

$17,580 

$293,000 

$210,304 

$6,517 

$10,000 

$537,401 

$438,680 

4 

$17,580 

$293,000 

$210,.304 

$6,517 

$10,000 

$537,401 

$409,981 

5 

$17,580 

$293,000 

$210,304 

$6,517 

$10,000 

$537,401 

$383,160 

Total 

$87,900 

$1,465,000 

$1,051,520 

$58,657 

$50,000 

$2,713,077 

$2,229,522 

We  identified  both  quantifiable  and  nonquantifiable  benefits  that  will  emerge  from 
establishing  the  recommended  training.  Although  quantifiable  benefits  appear  to  be 
more  tangible,  we  urge  OSD  and  military  department  leaders  not  to  discount  benefits 
that  cannot  be  quantified.  Such  benefits  can  be  just  as  or  even  more  important  than 
benefits  that  can  be  quantified  and  have  actual  dollar  value.  A  discussion  of  these 
benefit  types  follows. 


Quantifiable  Beneflts 

Although  training  is  commonly  considered  to  increase  efficiency,  customer  satisfac¬ 
tion,  employee  job  satisfaction,  and  morale,  few  organizations  have  attempted  to 
measure  the  benefits  of  training.  Studies  that  have  quantified  the  benefits  of  training 
report  a  wide  range  of  returns.  For  instance,  one  study  concluded,  based  on  a  sub¬ 
jective  measure,  that  employer-provided  training  raises  productivity  by  almost  16 
percent^  while  also  citing  results  from  a  Bell  Helicopter  massive  training  program 
that  resulted  in  a  productivity  increase  of  181  percent  (Laabs,  1997,  p.  9). 

Because  of  such  uncertainty  in  the  quantifiable  results  of  training,  we  make  what  we 
consider  to  be  conservative  estimates  regarding  the  process  improvements  resulting 
from  training.  Our  estimates  are  just  that — estimates.  Only  empirical  testing,  ex  post, 
can  prove  these  estimates  to  be  sound.  In  this  section  of  the  chapter,  we  discuss  the 
benefits  of  the  combined  training  initiatives,  rather  than  evaluating  them  separately, 
although  we  believe  that  the  classroom  training  is  likely  to  bring  about  the  vast  ma¬ 
jority  of  the  nonquantifiable  benefits  we  discuss  later  in  this  chapter. 

For  the  DES,  the  quantifiable  benefit  that  we  predict  training  will  produce  is  a  reduc¬ 
tion  in  medical  board  or  case  processing  time.  Ideally,  processing  time  reduction  es¬ 
timates  would  relate  specifically  to  detailed  hypotheses  regarding  how  training  will 
decrease  processing  time.  For  example,  we  hypothesize  the  following: 

•  Training  PEBLOs  will  reduce  the  number  medical  boards  the  PEB  sends  back  to 
the  medical  evaluation  phase  at  the  MTF  due  to  the  medical  board  being 
administratively  incomplete. 


®As  noted  in  Black  and  Lynch,  (1996),  p.  263. 
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•  Training  physicians  will  reduce  the  number  of  medical  boards  the  PEB  sends 
back  to  the  MTF  because  the  narrative  summary  is  medically  insufficient  and/ or 
will  reduce  the  number  of  instances  PEB  members  must  phone  MTF  physicians 
for  more  detailed  information  about  a  medical  board. 

•  Training  commanders  will  reduce  the  number  of  medical  boards  the  PEB  sends 
back  to  the  MTF  for  additional  nonmedical  assessment  information  and  reduce 
the  number  of  days  it  takes  a  commander  to  submit  the  nonmedical  assessment 
commander’s  letter. 

In  the  absence  of  the  baseline  data  for  these  detailed  steps  in  the  DES.^^  we  estimate 
changes  in  medical  board  processing  time  at  the  aggregate  levels  of  the  Medical 
Evaluation  Board  in  the  medical  evaluation  phase  and  the  PEB  in  the  physical  dis¬ 
ability  evaluation  phase.  We  assume  that  the  initial  year  (Year  0)  will  be  spent  devel¬ 
oping  training  programs  and  that  training  will  not  be  fielded  until  Year  1.  As  shown  in 
Table  7.5,  we  estimate  that  average  Medical  Evaluation  Board  processing  time^iwill 
improve  from  its  current  baseline  1.75-percent  change  in  the  first  year  self-directed 
computer-based  distance  training  is  introduced  to  a  2.5-percent  change  in  the  sec¬ 
ond  year.  We  then  assume  that  the  improvements  will  remain  at  a  constant  2.5-per¬ 
cent  rate  in  the  following  years;  in  other  words,  no  further  improvements  will  occur. 
Again,  this  improvement  figure  is  in  comparison  with  the  current  baseline. 

We  estimate  that  Physical  Evaluation  Board  processing  time^^  ^ill  improve  at  a  lesser 
rate — a  0.75-percent  change  in  the  first  year  and  a  1.25-percent  change  in  the  follow¬ 
ing  years.  We  estimate  lesser  changes  in  medical  board  processing  time  during  the 
first  year  that  training  is  fielded  because  we  suspect  that  training  may  be  phased  in 
gradually  rather  than  at  day  one  during  the  first  year  training  is  fielded.  If  all  partici¬ 
pants  are  trained  immediately,  our  estimates  for  the  first  year  of  training  may  under¬ 
state  the  change  in  medical  board  processing  time  and  subsequent  benefits.  We  as¬ 
sume  a  constant  reduction  in  processing  time  in  the  subsequent  years  because  it 
provides  a  more  conservative  estimate  of  the  benefits. 

Table  7.5 

Change  in  Average  Medical  Board  Processing  Time  Due  to  Training  Intervention 


Change  in  Average  Processing  Time  (in  Days) 


Year 

Phase  of  Process 

Percent 

Change 

Department 
of  the  Army 

Department 
of  the  Navy 

Department 
of  the  Air  Force 

Year  0 

All 

0 

0 

0 

0 

Year  1 

Medical  Evaluation  Board 

1.75 

0.8750 

1.155 

0.610 

Physical  Evaluation  Board 

0.75 

0.2175 

0.660 

0.138 

Additional 

Medical  Evaluation  Board 

2.50 

1.2500 

1.650 

0.875 

Years 

Physical  Evaluation  Board 

1.25 

0.3625 

1.100 

0.230 

all  of  the  military  departments  were  able  to  provide  us  with  such  detailed  process  information. 

l^Medical  Evaluation  Board  processing  time  extends  from  the  date  the  narrative  summary  is  dictated  to 
the  date  the  medical  board  is  received  by  the  Informal  PEB. 

^^PEB  processing  time  extends  from  the  date  the  Informal  PEB  receives  the  medical  board  to  the  date  of 
the  final  reviewing  authority’s  disposition  decision. 
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We  predict  greater  change  in  medical  board  processing  time  during  the  medical 
evaluation  phase,  in  part  because  so  many  factors  and  participants  influence  pro¬ 
cessing  time  in  that  phase.  For  instance,  the  medical  evaluation  phase  processing 
time  reflects  the  amount  of  time  it  takes  for  (1)  a  PEBLO  or  patient  administrator  to 
assemble  all  of  the  required  documents,  including  specialty  consults,  LOD  determi¬ 
nations,  nonmedical  assessments,  and  other  documents,  to  complete  a  medical 
board,  (2)  a  PEBLO  to  counsel  a  service  member;  (3)  a  commander  to  send  a  non¬ 
medical  assessment  commander’s  letter;  and  (4)  a  Medical  Evaluation  Board  to  pro¬ 
cess  a  medical  board. 

Although  we  assume  the  same  rate  of  change  for  all  of  the  military  departments,  the 
effect  of  this  change  produces  different  reductions  in  the  daily  measure  of  average 
medical  board  processing  time  due  to  differing  processing  time  baselines  that  exist 
among  the  military  departments  (see  Table  7.6). 

Based  on  these  estimates,  the  benefits  due  to  reduced  medical  board  processing  time 
will  save  the  military  departments  approximately  $15.17  million  over  the  five-year 
period  (see  Table  7.7).  Appendix  I  contains  cost-benefit  analysis  data  detailing  the 
quantifiable  benefits  for  each  of  the  military  departments. 

Every  day  that  is  cut  off  from  the  processing  time  of  an  active  duty  medical  board  re¬ 
sults  in  a  quantifiable  benefit  to  the  military  departments  as  the  time  to  replace  a 
member  (either  through  separation,  retirement,  or  a  return  to  duty)  who  is  not  per¬ 
forming  his  or  her  duty  is  reduced.  The  faster  the  DES  can  move  a  service  member 
through  the  system,  the  less  money  or  time  the  military  departments  pay  for  an  un¬ 
manned  billet.  We  monetize  this  benefit  by  calculating  the  average  daily  pay  for  the 


Table  7.6 

Average  Processing  Time,  FY1999 


Average  Processing  Time  (in  Days) 


Department  Department 

Phase  of  Process _ oftheArmy _ of  the  Navy 

Medical  Evaluation  Board  50  66 

fi? _  29  88 


Department  of 
the  Air  Force 


35 

18 


Table  7.7 

Quantifiable  Benefits  for  All  Military  Departments 


Medical  Evaluation  Board _  Physical  Evaluation  Board 


Year 

TDRLFit 

TDRL 

Separate 

Active 

TDRL  Fit 

TDRL 

Separate 

Active 

0 

1 

$17,940 

$41,089 

$1,755,679 

$5,434 

$17,787 

$  747,775 

2 

$17,940 

$58,727 

$2,509,090 

$9,057 

$29,646 

$1,246,291 

3 

$17,940 

$58,727 

$2,509,090 

$9,057 

$29,646 

$1,246,291 

4 

$17,940 

$58,727 

$2,509,090 

$9,057 

$29,646 

$1,246,291 

5 

$17,940 

$58,727 

$2,509,090 

$9,057 

$29,646 

$1,246,291 

Total 

$84,312 

$275,995 

$11,792,037 

$41,661 

$136,370 

$5,732,940 

Total  Year  NPV  Benefit 

$2,580,316  $2,494,485 
$3,870,750  $3,497,192 
$3,870,750  $3,268,404 
$3,870,750  $3,054,583 
$3,870,750  $2,854,750 
$18,063,316  $15,169,414 
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average  member  in  the  system^^  and  multiplying  that  average  daily  pay  by  the  aver¬ 
age  reduction  in  processing  time  and  the  number  of  cases  completed  by  the  system 
each  year. 

As  shown  in  Table  7.7,  we  monetized  this  benefit  differently  for  members  who  are  on 
the  temporary  disability  retired  list  (TDRD.i^For  members  on  active  duty,  a  one-day 
reduction  in  processing  time  equates  to  a  one-day  reduction  of  pay,  but  for  indi¬ 
viduals  being  processed  through  the  DBS  who  are  on  the  TDRL,  the  cost  savings  from 
taking  the  individual  off  the  TDRL  a  day  earlier  depends  upon  the  final  adjudication 
of  the  case. 

Individuals  who  are  continued  on  the  TDRL  or  who  are  permanently  retired 
(assuming  no  change  in  the  final  disability  rating)  receive  the  same  payment  after 
processing  as  they  do  during  processing.  As  a  result,  a  reduction  in  processing  time 
does  not  affect  retired  pay  disbursements.  TDRL  cases  that  result  in  a  fitness 
determination  reduce  disbursements  by  the  daily  TDRL  payment  (50  percent  of  base 
pay)  for  each  day  processing  time  is  reduced.  Individuals  who  are  found  unfit  and 
separated  with  severance  pay  reduce  disbursements  by  the  daily  TDRL  payments 
minus  the  cost  of  paying  the  individuals  severance  pay  one  day  earlier.  Due  to  the 
marginal  rate  of  time  preference, a  dollar  today  is  worth  more  than  a  dollar  tomor¬ 
row.  Based  on  the  7-percent  discount  rate  used  in  this  analysis,  the  cost  of  paying  an 
individual  an  average  severance  payment  ($30,000)  one  day  early  equates  to  $5.35. 

We  assumed  that  10  percent  of  the  TDRL  cases  would  be  found  fit,  40  percent  would 
be  separated  with  severance  pay,  and  50  percent  would  be  permanently  retired  or  re¬ 
tained  on  the  TDRL  (with  no  change  in  the  disability  rating). 

As  is  evident  in  Table  7.7,  we  do  not  forecast  future  caseloads  or  manning  levels, 
rather  we  base  our  calculations  on  constant  caseload  and  manning  levels,  which  are 
based  on  the  most  recent  caseload  data  provided  to  us  by  the  military  departments 


^^We  assume  that  the  average  rank  of  the  members  being  processed  through  the  DES  is  E-5.  Based  on 
caseload  by  rank  data  we  received  from  the  Department  of  the  Army,  we  believe  this  is  a  sound  and 
conservative  estimate. 

^"^We  would  have  liked  to  include  cost  information  for  reservists  as  well,  but  not  all  of  the  military 
departments  were  able  to  provide  us  this  data. 

l^TDRL  payments  are  made  from  the  Department  of  Defense  Retirement  Fund  and,  therefore,  do  not 
directly  affect  the  defense  budget.  Although  the  department  contributes  an  amount  each  month  to  the 
fund  to  pay  for  future  benefits  earned  in  that  month,  the  changes  that  we  recommend  in  this  report  are 
unlikely  to  affect  the  calculation  of  that  accrual  charge.  However,  the  changes  will  have  an  effect  on 
disbursements  from  the  fund  and,  therefore,  from  the  federal  government  (even  though  the  department 
will  not  see  the  impact  in  its  budget).  We  capture  that  larger  impact  in  our  analysis. 

^^Members  receive  temporary  disability  retired  pay  equal  to  base  pay  multiplied  by  the  rated  percent  of 
disability  (or  retired  pay  equal  to  base  pay  multiplied  by  2.5  percent  for  each  year  of  service,  whichever  is 
greater),  but  the  retired  pay  cannot  fall  below  50  percent  of  base  pay  or  be  more  than  75  percent  of  base 
pay.  Our  TDRL  calculations  are  conservative,  based  upon  the  minimum  payment  a  member  will  receive 
while  on  the  TDRL. 

^^The  “marginal  rate  of  time  preference”  is  an  economic  concept.  A  dollar  today  is  worth  more  than  a 
dollar  tomorrow  for  everyone.  For  example,  most  people  would  be  unwilling  to  lend  someone  $100  today 
in  return  for  $100  next  year.  People  generally  value  $100  today  more  than  the  promise  of  $100  next  year, 
even  if  they  are  certain  that  it  will  be  repaid  and  there  will  be  no  inflation,  because  of  their  preference  to 
consume  sooner  rather  than  later.  The  marginal  rate  of  time  preference  is  the  rate  at  which  individuals 
make  marginal  trade-offs  in  consuming  now  versus  later. 
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(see  Table  7.8).  If  primary  participant  numbers  increase  and  decrease  proportionally 
with  an  increase  or  decrease  in  caseload,  the  ratio  of  costs  to  benefits  will  remain  the 
same.  If  primary  participant  numbers  do  not  change  proportionally  to  the  change  in 
caseload,  the  ratio  of  costs  and  benefits  we  estimate  will  not  hold.  For  instance,  if  the 
Department  of  the  Navy  caseload  increases  by  2,000  cases  per  year  and  no  primary 
participants  are  added  to  respond  to  the  increase  in  caseload,  it  is  likely  that  process¬ 
ing  time  would  not  decrease  at  the  rate  we  estimate. 

We  believe  the  DoD-wide  training  program  recommended  in  this  report  will  displace 
much  of  the  current  training  provided  by  the  military  departments.  However,  we 
foresee  that  the  military  departments  will  establish  a  new  kind  of  disability  evalua¬ 
tion  training  that  focuses  on  executing  unique  departmental  human  resource  and 
administrative  policies  and  procedures  as  well  as  developing  other  abilities,  charac¬ 
teristics,  and  behaviors  required  to  produce  desired  results  on  the  job  (for  instance, 
developing  counseling  techniques).  As  a  result,  we  did  not  compute  any  savings  from 
the  military  departments  reducing  their  current  training  levels,  which  may  result  in  a 
conservative  estimate  of  benefits. 

Nonquantifiable  Benefits 

In  addition  to  savings  from  reductions  in  medical  board  processing  time,  training 
produces  a  number  of  nonquantifiable  benefits  for  the  military  departments  and  the 
OSD  (see  Table  7.9).  Two  such  benefits  are  the  increase  in  consistent  application  of 
disability  policy  within  the  military  departments  and  the  increase  in  consistent  ap¬ 
plication  of  disability  policy  among  the  military  departments.  Although  both  benefits 
are  important  to  the  OSD,  the  military  departments  only  benefit  from  consistent 
application  of  policy  within  the  individual  military  department.  Both  the  self- 
directed  computer-based  distance  training  and  classroom  training  result  in  more- 
consistent  application  of  disability  policy,  but  the  classroom  training  produces  the 
greatest  gains  in  the  consistent  application  of  policy. 

In  Chapter  5,  we  noted  that  PEB  members,  PEB  approving  authorities,  and  post-PEB 
appellate  review  board  members,  in  particular,  stand  to  benefit  from  collaboration 
with  peers  on  how  to  uniformly  apply  the  rules,  procedures,  and  other  considera¬ 
tions  in  determining  fitness  ratings,  VASRD  codes  including  analogous  codes,  and 
disability  ratings.  Likewise,  Medical  Evaluation  Board  members  and  approving  au¬ 
thorities  stand  to  benefit  from  collaboration  with  peers  on  how  to  apply  disciplined 
medical  retention  standards  uniformly  within  military  departments  as  well  as  the 


Table  7.8 

Number  of  Cases  (Medical  Boards)  per  Year 


Department  of  the 

Department  of  the 

Department  of  the 

Case  Type 

Army  (FY99) 

Navy  (FY98) 

Air  Force  (TO8) 

Active 

7,564 

9,125 

2,954 

TDRL 

1,472 

2,140 

833 
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Table  7.9 

Nonquantifiable  Benefits  from  Training 


Benefits 

Computer-Based  Distance 
Training 

Classroom  Training 

OSD 

Military 

Departments 

OSD 

Military 

Departments 

Consistent  application  of  disability 

X 

X 

XX 

XX 

policy  within  the  military 

departments 

Consistent  application  of  disability 

X 

XX 

policy  among  the  military 

departments 

Increased  unit  readiness 

X 

X 

X 

X 

Increased  customer  satisfaction 

X 

X 

X 

X 

Increased  job  satisfaction 

X 

X 

X 

X 

NOTE:  X  =  nonquantifiable  benefits;  XX  =  even-greater  nonquantifiable  benefits,  compared  with  the  al¬ 
ternative  training  delivery  method. 


kind  and  level  of  medical  details  required  in  medical  boards  for  PEB  members  to  ad¬ 
judicate  cases.  Through  this  collaboration,  we  expect  that  disability  policy  will  be 
more  uniformly  applied  both  within  and  among  the  military  departments. 

A  nonquantifiable  benefit  accrues  to  the  OSD  and  the  military  departments  due  to 
increased  unit  readiness.  Although  we  already  captured  part  of  this  benefit  in  the 
quantifiable  benefits,  we  believe  there  is  an  additional  nonquantifiable  benefit  of  in¬ 
creased  unit  readiness  that  exceeds  the  military  department’s  cost  for  a  service 
member.  In  other  words,  the  sum  of  the  whole  (a  complete  and  ready  unit)  is  greater 
than  the  sum  of  its  parts  (all  the  individual  members  of  the  unit). 

Training  also  increases  customer  satisfaction.  In  Chapter  6,  we  identify  the  service 
members  being  processed  through  the  DES  and  the  military  services  as  customers. 
We  hypothesize  that  training  increases  service  member  satisfaction  by  reducing  case 
variability  and  strengthening  confidence  in  the  system,  and  it  increases  military  ser¬ 
vice  satisfaction  by  lowering  the  total  cost  of  the  system  and  reducing  the  time  to  re¬ 
place  “broken”  members. 

Furthermore,  training  results  in  increased  job  satisfaction  among  all  the  primary 
participant  populations  trained.  Developing  greater  knowledge  and  common  un¬ 
derstanding  of  the  system  plus  reducing  the  number  of  reworks  better  enables  pri¬ 
mary  participants  to  produce  desired  on-the-job  results,  which  leads  to  pride  in 
performance  and  increased  job  satisfaction. 

TRAINING  RESULTS 

Over  the  five-year  time  frame  of  the  intervention,  the  quantitative  benefits  outweigh 
the  costs  by  approximately  $2.39  million.  As  shown  in  Table  7.10,  during  the  devel¬ 
opment  year,  costs  outweigh  benefits,  assuming  Year  0  will  be  dedicated  to  develop¬ 
ing  the  training  packages.  During  Year  1  of  the  training,  costs  again  outweigh  the 
benefits.  However,  the  benefits  outweigh  the  costs  in  each  of  the  remaining  years  (as 
reduced  costs  reflect  the  impact  of  shorter  processing  times). 
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In  addition  to  the  $2.39  million  in  benefits  that  training  produces,  training  also  pro¬ 
duces  the  nonquantifiable  benefits  of  consistent  policy  application,  increased  job 
and  customer  satisfaction,  and  increased  unit  readiness. 

If  this  analysis  understated  the  change  in  medical  board  processing  time  and  the  ac¬ 
tual  changes  looked  instead  like  those  in  Table  7.11, the  net  present  value  of  the 
benefits  would  equate  to  approximately  $27.3  million,  causing  the  total  net  present 
value  of  the  training  sessions  to  be  approximately  $14.5  million. 

On  the  other  hand,  if  one  rejects  our  hypothesis  that  training  results  in  process  im¬ 
provements  (which  we  believe  is  a  strong  hypothesis)  and  assumes  that  no  quantifi¬ 
able  benefits  accrue  from  the  training,  it  would  cost  approximately  $12.78  million 
over  six  years  (NPV  costs  from  Years  0  through  5)  to  obtain  the  nonquantifiable 
benefits  we  identified. 


Table  7.10 

Net  Present  Value  Quantifiable  Costs  and  Benefits 


Year  NPV  Costs 

NPV  Benefits 

NPV  Total 

0 

$6.%,419 

— 

($656,419) 

1 

$2,997,861 

$2,494,485 

($503,376) 

2 

$2,518,245 

$3,497,192 

$978,947 

3 

$2,353,500 

$3,268,404 

$914,904 

4 

$2,199,532 

$3,054,583 

$855,050 

5 

$2,055,638 

$2,854,750 

$799,113 

Total  $12,781,195 

$15,169,414 

$2,388,219 

Table  7. 11 

Alternate  Change  in  Average  Medical  Board  Processing  Time  Due  to  Training  Intervention 

Change  in  Average  Processing  Time  (in  Days] 

Phase  of  Process 

Percent 

Department  of 

Department  of  Department  of 

Year 

Change 

the  Army 

the  Navy  the  Air  Force 

YearO 

All 

0 

0 

0  0 

Year  1 

Medical  Evaluation  Board 

2.50 

1.2500 

1.65  0.8715 

Physical  Evaluation  Board 

2.25 

0.3625 

1.10  0.2300 

Additional 

Medical  Evaluation  Board 

5.00 

2.5000 

3.30  1.7430 

Years 

— 

Physical  Evaluation  Board 

2.25 

0.6525 

1.98  0.4140 

1  ft 

We  believe  the  alternate  estimate  shown  in  Table  7.1 1  also  has  merit. 


Chapter  Eight 

CONCLUSIONS 


This  concluding  chapter  frames  our  recommendations  within  the  context  of  the 
central  theme  of  this  report.  This  chapter  also  offers  some  observations  that  suggest 
further  efforts  to  complement  those  recommendations. 

STUDY  OBJECTIVES,  APPROACH,  AND  RECOMMENDATIONS 

Improving  the  performance  of  the  Disability  Evaluation  System  is  the  central  theme 
of  this  report.  This  theme  emerges  from  and  encompasses  the  initial  study  objective 
of  identifying  and  recommending  changes  to  the  training  provided  to  the  primary 
participants  of  the  DES  to  ensure  consistent  application  of  disability  policy,  both 
across  and  within  the  military  departments. 

We  addressed  the  study  objective  with  an  issues-based,  bottom-up  approach.  We 
started  by  reviewing  the  OSD  and  military  department  policy  documents  that  govern 
the  operation  of  the  DES,  attending  the  various  military  departments’  DES  training 
events,  and  interviewing  numerous  and  diverse  primary  participants  from  all  the 
military  departments.  Based  on  these  sources  of  information,  we  recorded  specific 
differences  in  terms  of  how  policy  was  understood,  policy  application,  military 
department  DES  operations,  availability  of  system  information,  and  we  identified 
problems. 

We  restated  the  differences  among  the  military  department  DESs  and  the  significant 
problems  that  were  identified  as  “issues”  affecting  the  consistent  application  of  dis¬ 
ability  policy  across  and  sometimes  within  the  military  departments.  We  then  formu¬ 
lated  desired  results — what  one  would  observe  if  an  issue  were  resolved — for  those 
issues,  which  led  to  a  set  of  recommended  actions  grouped  into  ten  categories  of  in¬ 
terventions  (one  of  which  focuses  on  training,  the  study  objective)  for  eliminating  the 
undesirable  differences  and  the  problems. 

The  ASD/FMP,  in  coordination  with  the  ASD/HA  and  the  ASD/RA,  can  implement 
these  recommendations  (described  in  Chapter  4)  immediately,  and  doing  so  will  de¬ 
cidedly  move  the  DES  toward  more-consistent  application  of  disability  policy. 
However,  a  broader  perspective — focused  on  overall  system  performance — promises 
a  significantly  more  far-reaching  and  profound  impact. 

In  particular,  we  could  have  developed  the  DoD  training  recommendations  based 
solely  on  the  evidence  we  uncovered  during  our  search  for  differences  in  policy  in- 
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terpretations  and  applications,  operation  of  the  military  department  DESs,  and  sys¬ 
tem  information  available  to  inform  decisions.  In  fact,  based  on  subjective  perfor¬ 
mance  issues  as  reported  by  primary  participants,  this  issues-based,  bottom-up  ap¬ 
proach  identified  a  limited  set  of  DES  topics  as  the  basis  for  a  narrowly  focused 
training  intervention. 

This  issues-based,  bottom-up  training  intervention,  presented  in  Chapter  4  among 
nine  other  interventions,  focuses  primarily  on  improving  system  efficiency  to  resolve 
a  common  set  of  continuing  problems  reported  by  primary  participants  across  the 
military  departments.  As  is  typical  for  an  issues-based,  bottom-up  training  needs  as¬ 
sessment,  the  resulting  narrowly  focused  training  intervention  has  little  or  no  regard 
for  achieving  any  specific,  uniformly  agreed-upon  overall  system  purpose  or  desired 
outcomes  and  improving  overall  system  performance. 

Developing  and  delivering  training  that  is  focused  on  a  relatively  limited  set  of  DES 
topics  would  clearly  result  in  more-consistent  application  of  disability  policy. 
However,  such  a  recommendation  would  not  be  well  grounded  in  training  theory 
and  application,  nor  in  performance  and  strategic  management  theory  and  applica¬ 
tion  for  that  matter.  In  addition,  this  issues-based,  bottom-up  approach  would  not 
take  full  advantage  of  training  as  a  key  intervention  to  improve  overall  system  per¬ 
formance.  We  decided  that  a  broader,  more-robust  approach  to  developing  a  DoD 
training  intervention  was  necessary. 

Rather  than  starting  with  current  problems  identified  by  primary  participants,  a 
broader  approach  begins  with  an  analysis  of  learner  needs,  referred  to  as  perfor¬ 
mance  analysis.  Performance  analysis  determines  if  a  performance  problem  related 
to  a  lack  of  knowledge  or  skill  exists.  The  foundation  for  performance  analysis  is  a 
published  statement  of  organizational  intent — expressed  in  measurable  terms  such 
as  goals,  objectives,  or  outcomes — that  allows  everyone  in  the  organization  to  focus 
on  and  take  action  to  achieve  the  same  stated  intent. 

In  the  case  of  the  DoD  DES,  we  propose  that  the  foundation  for  performance  analysis 
is  a  published  statement  of  system  purpose  and  desired  system  outcomes  that  ap¬ 
plies  across  military  departments,  as  described  in  Chapter  3.  Another  critical  compo¬ 
nent  of  this  foundation  is  a  method  for  monitoring  system  performance  across  mili¬ 
tary  departments  over  time,  as  described  in  Chapter  6.  Actual  recorded  performance 
results  point  to  potential  problems  or  areas  that  require  investigation.  Chances  are 
good  that  some  of  these  problems  suggest  a  lack  of  knowledge  or  skill  within  a  pri¬ 
mary  participant  population.  This  need  for  additional  knowledge  or  skill  develop¬ 
ment  then  serves  as  a  basis  for  developing  additional  training— which  includes 
preparation  of  learner-centered  objectives,  developing  content,  writing  test  ques¬ 
tions,  and  such — or  modifying  existing  training  and  assessing  training  effectiveness. 

The  system  purpose  and  desired  outcomes,  in  turn,  shape  the  competencies  required 
for  primary  participant  populations  to  achieve  desired  on-the-job  results.  Stated 
performance  competencies  are  another  prerequisite  to  developing  effective  DoD 
training  targeted  to  the  needs  of  diverse  primary  participant  population  clusters 
across  military  departments.  As  noted  in  Chapter  3,  each  military  department  has 
formulated  a  unique  statement  of  intent  for  operating  its  DES.  Aside  from  OSD  policy 
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language,  we  found  no  evidence  that  statements  of  desired  system  outcomes  across 
military  departments  exist  to  help  focus  decisionmaking  in  regard  to  consistent  ap¬ 
plication  of  disability  policy  or  improvement  of  overall  system  performance. 

Consequently,  we  employed  a  purpose-driven,  top-down  approach  to  developing  the 
comprehensive  training  recommendations  presented  in  Chapter  5  that  is  more- 
robust  than  the  bottom-up  approach.  We  started  with  a  set  of  desired  system 
outcomes  that  explicitly  states  the  intended  results  of  operating  the  DBS  to  achieve 
its  stated  purpose,  as  proposed  in  Chapter  3. 

Given  the  existing  assignment  practices  and  job  designs  for  primary  participant 
populations,  we  translated  the  proposed  statements  of  desired  system  outcomes  into 
statements  that  describe  the  major  activities  that  the  primary  participant  popula¬ 
tions  must  be  able  to  perform  to  achieve  those  desired  outcomes.  These  statements 
shaped  the  formulation  of  the  proposed  competencies  that  primary  participants 
need  in  order  to  achieve  both  the  desired  on-the-job  results  and  the  overall  system 
outcomes.  These  competencies  then  pointed  to  a  DoD  training  emphasis  on  apply¬ 
ing  specific  bodies  of  knowledge  and  skills  across  the  military  departments. 

We  identified  107  DBS  topics,  including  27  identified  using  the  bottom-up,  issues- 
driven  approach  described  in  Chapter  4,  as  the  basis  for  developing  the  training 
packages  described  in  Chapter  5.  Based  on  our  assessment  of  the  competencies  re¬ 
quired  for  primary  participant  populations  to  produce  desired  on-the-job  results  to 
achieve  overall  desired  DBS  outcomes,  we  organized  the  DBS  topics  into  specific 
bodies  of  knowledge  required  by  the  primary  participant  populations. 

We  observed  that  five  groups,  or  clusters,  of  primary  participant  populations  re¬ 
quired  essentially  the  same  body  of  knowledge  and  skills  to  produce  the  desired  on- 
the-job  results.  We  organized  those  bodies  of  knowledge  into  five  training  packages 
targeted  to  the  specific  performance  needs  of  five  primary  participant  population 
clusters.  These  five  training  packages  constitute  the  course  content  for  a  complete 
DoD  disability  evaluation  training  program,  which  is  presented  in  Chapter  5. 

To  assess  the  effectiveness  of  the  training  content  and  its  delivery,  the  OSD  needs  a 
comprehensive  system  for  monitoring  system  performance.  Without  such  a  system, 
the  OSD  will  not  be  able  to  evaluate  the  effectiveness  of  its  training  program  or  the 
actions  to  implement  other  interventions.  In  addition,  although  DoD  training  and  a 
system  for  monitoring  overall  system  performance  are  probably  the  most  effective 
means  of  improving  the  performance  of  the  DBS,  other  interventions,  such  as  those 
described  in  Chapter  4  that  are  based  on  our  bottom-up  analysis,  also  contribute  to 
improving  system  performance. 

Consequently,  we  recommend  that  the  OSD  develop  a  management  information 
system  capable  of  assessing  DBS  performance  on  a  continuing  basis  in  order  to 
identify  areas  for  improvement  and  develop  specific  plans  for  achieving  those  im¬ 
provements. 

In  summary,  we  recommend  the  following  three  major  interrelated  actions  to  im¬ 
prove  the  performance  of  the  DBS  over  time: 
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1.  The  ASD/FMP  directs  the  Director  of  Officer  and  Enlisted  Personnel  Management 
to  consult  with  a  small  group  of  experienced  DES  experts  representing  the  military 
departments’  PEBs  and  Office  of  the  Surgeons  General  to  produce  recommenda¬ 
tions  upon  which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and  the 
ASD/RA,  can  decide  upon  a  system  purpose  and  desired  outcomes. 

2.  The  ASD/FMP  directs  the  Director  of  Officer  and  Enlisted  Personnel  Manage¬ 
ment  to 

•  develop  and  monitor  knowledge-based  training  in  which  the  content  focuses  on 
the  suggested  list  of  DES  topics  that  constitute  the  specific  body  of  knowledge  for 
each  of  five  primary  participant  population  clusters  to  achieve  the  desired  DES 
outcomes 

•  deliver  this  standardized,  knowledge-based  training  "just-in-time”  through  self- 
directed  computer-based  distance-training  packages,  each  targeting  a  particular 
primary  participant  population  cluster,  via  a  DoD  Web  site  devoted  to  disability 
evaluation  training  that  is  accessible  by  all  primary  participants 

•  supplement  this  self-directed  computer-based  distance  training  with  classroom 
training  targeted  to  two  population  clusters;  this  training  focuses  on  applying 
knowledge  of  a  particular  set  of  DES  topics  to  develop  the  skills  necessary  to 
evaluate  and  adjudicate  cases  consistently  across  and  within  the  military 
departments— a  primary  determinant  of  consistent  application  of  disability 
policy. 

3.  The  ASD/FMP,  after  consulting  on  the  information  needs  of  the  ASD/HA  and  the 
ASD/RA,  directs  the  Director  of  Officer  and  Enlisted  Personnel  Management  to 
develop  and  maintain  a  comprehensive  management  information  system  capable 
of  monitoring  relevant  performance  measures  (as  they  apply  to  both  the  active 
and  Reserve  components)  that  enables  leaders  to  assess  and  analyze  DES 
performance  and  take  action  to  continually  improve  that  performance. 

In  Chapter  3,  we  propose  a  specific  stated  purpose  for  the  DES;  however,  it  is  only  a 
suggestion  based  on  our  analysis.  We  chose  a  top-down,  purpose-driven  approach  in 
order  to  design  and  apply  a  method  for  developing  the  proposed  training  interven¬ 
tion  and  a  set  of  metrics  for  use  in  a  management  information  system  intervention. 
Chapters  5  and  6  describe  this  method  and  the  metrics.  To  accomplish  the  second 
and  third  recommended  actions,  we  strongly  urge  the  department  to  develop  its  own 
stated  purpose  for  the  DES  and  desired  DES  outcomes,  and  then  apply  the  method¬ 
ology  described  in  Chapters  5  and  6  using  the  stated  purpose  that  results  from  the 
first  recommendation  listed  in  this  section. 

The  direction-setting  statements— the  purpose  and  desired  outcomes — do  not  have 
to  be  100-percent  perfect;  the  important  thing  is  to  establish  them  in  order  to  de¬ 
velop  and  deliver  a  DoD-wide  DES  training  intervention  that  begins  to  positively  im¬ 
pact  the  consistent  application  of  disability  policy  and  overall  system  performance. 
Likewise,  it  is  preferable  to  quickly  launch  a  comprehensive  training  intervention  be¬ 
lieved  to  be  an  85-  or  90-percent  solution  and  simultaneously  commit  to  making 
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continuous  improvements  based  on  the  performance  results  and  lessons  learned 
from  the  intervention. 

Implementing  a  comprehensive  training  intervention  and  the  associated  manage¬ 
ment  information  system  intervention  will  require  considerable  time  and  effort  be¬ 
fore  they  produce  measurable  performance  results;  therefore,  they  should  be  initi¬ 
ated  immediately. 

SYSTEMS  PERSPECTIVE  OF  THE  DISABILITY  EVALUATION  SYSTEM 

As  stated  earlier,  consistent  with  the  initial  study  objective,  this  report  focuses  on 
identifying  and  resolving  problems  that  contribute  to  inconsistent  application  of  dis¬ 
ability  policy  within  the  DES  (see  Chapter  4).  Although  achieving  a  greater  degree  of 
consistency  in  the  application  of  disability  policy  increases  the  value  of  the  DES,  it  is 
desirable,  and  possible,  to  increase  its  value  even  more  by  relentlessly  focusing  on 
the  end  of  continuously  increasing  the  efficiency  of  DES  operations  and  overall  sys¬ 
tem  performance.  The  interventions  recommended  in  Chapters  4,  5,  and  6  provide 
the  foundation  upon  which  to  establish  this  focus.  Developing  a  systems  perspec¬ 
tive — routinely  monitoring  data  to  provide  performance-based  feedback  that  enables 
decisionmakers  to  “see”  the  system  interconnections  that  both  cause  and  ejfect  per¬ 
formance  results— is  the  means  to  this  end.  Data  are  the  essential  resource  for  con¬ 
tinuously  improving  efficiency  of  operations  and  overall  system  performance. 

Primary  participants  from  all  the  military  departments  expressed  considerable  frus¬ 
tration  with  the  substantial  amount  of  detailed  information  that  they  record  and  re¬ 
port  through  various  systems  without  knowing  how  it  is  used.  Each  MTF  and  military 
department  PEB  collect  data,  but  to  what  end? 

The  military  departments  report  only  the  medical  board  processing  timeliness  mea¬ 
sures  to  the  OSD  even  though  they  report  various  other  data  within  their  respective 
departments.  In  other  words,  the  OSD  currently  relies  solely  on  medical  board  pro¬ 
cessing  times  to  assess  the  performance  of  the  DES.  Furthermore,  primary  partici¬ 
pants  perceive  that  reporting  processing  timeliness  measures  to  the  OSD  and  meet¬ 
ing  (or  missing)  the  OSD-imposed  timeliness  standard  is  not  really  important.  They 
view  the  standard  as  merely  an  arbitrary  measure  with  no  real  accountability  at¬ 
tached  to  it. 

Most  primary  participants  that  we  interviewed  appeared  not  only  interested  in  im¬ 
proving  system  operating  efficiency  but  also  committed  to  it.  They  want  to  produce 
results  that  make  them  proud  of  what  they  do  every  day.  They  do  not  object  to 
collecting  data  when  they  understand  how  the  data  add  value  to  system  operations, 
but  they  do  object  to  collecting  data  just  for  its  own  sake.i 

Some  primary  participants  appear  to  be  in  the  unenviable  position  of  collecting  ev¬ 
ery  possible  data  element  “in  case  someone  asks  for  it.”  Yet,  when  we  asked  for  data 


^The  military  departments  generate  the  data  collection  requirements  for  essentially  all  data  collected 
other  than  data  related  to  timeliness. 
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on  the  number  of  medical  boards  that  are  returned  to  MTFs  due  to  insufficient  medi¬ 
cal  documentation  in  the  narrative  summary  and  the  average  number  of  days  it  takes 
for  a  medical  board  to  be  returned  to  an  Informal  PEB  after  it  is  sent  back  to  an  MTF 
for  insufficiency,  the  data  were  not  available  from  all  the  military  departments. 

Likewise,  although  the  military  departments  provided  data  we  requested  on  the  per¬ 
centage  of  service  members  referred  to  the  PEB  who  were  subsequently  found  fit  and 
returned  to  duty  and  the  percentage  of  Informal  PEB  decisions  service  members  ac¬ 
cepted  or  appealed,  fulfilling  the  request  seemed  to  take  an  excessive  amount  of 
time,  suggesting  that  the  military  departments  do  not  routinely  query  and  monitor 
this  data.  Similarly,  the  Office  of  the  ASD/RA  cannot  monitor  aspects  of  DES  perfor¬ 
mance  as  it  pertains  to  Reserve  component  members.  Even  so,  in  many  instances, 
significant  resources  are  devoted  to  data  collection. 

Deciding  on  a  purpose  statement  that  articulates  the  fundamental  reason  the  DES 
exists  and  a  set  of  desired  results  from  operating  the  system  to  achieve  that  purpose 
(the  desired  outcomes)  positions  the  ASD/FMP,  the  ASD/HA,  and  the  ASD/RA  to  de¬ 
termine  which  performance  measures — outcome  measures,  output  measures,  input 
measures,  or  metrics  best  assess  system  performance.  This  purpose-driven,  top- 
down  approach  will  likely  call  into  question  the  value  of  some  data  currently  being 
collected,  while  also  affirming  the  value  of  some  data  elements  that  help  assess  sys¬ 
tem  performance  and  flagging  others  that  are  collected  merely  out  of  habit  or  just  in 
case  someone  asks  for  them. 

Positioning  the  Assistant  Secretaries  to  determine  which  performance  measures  best 
assess  DES  is  just  one  example  of  how  a  shared  system  purpose  and  set  of  desired 
outcomes  will  add  value  to  the  DES.  The  DES  purpose  and  desired  outcomes  become 
the  focal  point  upon  which  to  base  complex  decisions  and  take  action  to 
continuously  improve  system  performance.  The  purpose  and  desired  outcomes  are 
the  basis  for  determining  exactly  what  to  measure  to  assess  overall  system  per¬ 
formance  in  a  way  that  is  meaningful  and  useful  to  those  accountable  for  DES  per¬ 
formance:  the  ASD/FMP,  the  ASD/HA,  and  the  ASD/RA;  Secretaries  of  the  military 
departments;  the  Surgeons  General;  MTF  commanders;  and  PEB  approving 
authorities. 

Monitoring  system  performance  produces  data,  the  essential  resource  for  continu¬ 
ously  improving  efficiency  of  operations  and  overall  system  performance.  It  also  po¬ 
sitions  the  OSD  to  champion  a  systems  perspective  that  focuses  on  emerging  issues 
or  unwarranted  variations  in  policy  application  and  the  dynamics  within  the  system 
that  cause  them. 

Developing  a  system-performance  perspective”^  among  primary  participants  and 
those  accountable  for  the  overall  performance  of  the  DES  enables  continuous  col¬ 
laborative  investigation  of  system  performance  based  on  feedback  from  all  sources 
(for  example,  from  primary  participant  satisfaction  surveys,  surveys  of  service  mem- 


system-performance  perspective  is  one  focused  on  monitoring  and  improving  system  performance. 
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bers  referred  to  the  DBS,  performance  measures — outcome,  output,  and  input,  and 
General  Accounting  Office  [GAO]  and  Inspector  General  [IG]  reports). 

A  system-performance  perspective  enables  primary  participants  and  those  respon¬ 
sible  for  overall  system  performance  to  “see”  the  pattern  of  interrelationships  within 
the  system  that  causes  and  ajfects  performance  and  therefore  helps  target  or  focus 
interventions  to  improve  system  performance.  The  ability  to  see  the  interrelation¬ 
ships  within  the  DBS  reveals  a  variety  of  areas  or  leverage  points,  some  high-leverage 
and  some  low-leverage,  that  may  benefit  from  interventions  to  improve  system  per¬ 
formance.  A  system-performance  perspective  enables  primary  participants  and 
those  responsible  for  overall  system  performance  to  recognize  the  impact  and  trade¬ 
offs  of  various  interventions  to  improve  that  performance. 

future  investigations  based  on  a  system-performance  perspective  may  lead  to  a  host 
of  system  interventions  focused  on  identified  leverage  points,  such  as  developing 
better  measures  of  overall  system  performance;  revising  DBS  policy;  developing  new 
sources  of  information;  changing  the  process;  or  implementing  training  interven¬ 
tions,  such  as  revising  the  knowledge-based  training,  introducing  new  knowledge- 
based  training,  or  introducing  a  new  method  of  delivery. 

Ideally,  a  system  performance  perspective  requires  a  management  information  sys¬ 
tem  that  monitors  DBS  performance  at  DoD,  military  department,  and  MTF  levels. 
The  management  information  system  measures  actual  results  compared  with  de¬ 
sired  results  and,  thereby,  focuses  training  and  other  interventions  on  closing  the  gap 
between  desired  outcomes  and  actual  outcomes. 

OTHER  APPLICATIONS  OF  THE  ISSUES-BASED  AND  PURPOSE-DRIVEN 
APPROACHES 

Although  this  report  focuses  on  the  DBS,  the  issues-based,  bottom-up  approach  and 
the  purpose-driven,  top-down  approach  applied  in  this  study  are  applicable  to  other 
components  of  the  DBS  and  other  systems  for  which  the  OSD  is  responsible. 

Applying  the  two  approaches  discussed  in  this  report— particularly  the  purpose- 
driven,  top-down  approach  presented  in  Chapter  3 — could  substantially  improve  the 
effectiveness  of  two  key  components  of  the  DBS  that  are  pertinent  to,  although  not 
within  the  scope  of,  this  study:  management  of  the  TDRL  and  application  of  the  DBS 
to  the  Reserve  component. 

Many  primary  participants  referred  to  problems  in  the  management  of  the  TDRL. 
The  military  departments  vary  in  their  ability  to  track  service  members  placed  on  the 
TDRL,  the  timeliness  of  the  evaluation  of  a  service  member  placed  on  the  TDRL,  and 
the  attention  they  give  to  the  TDRL.  Although  representing  only  about  5  percent  of 
disability  retired  pay  disbursements,  in  any  given  year,  approximately  9,000  service 
members  are  on  the  TDRL  (out  of  a  total  of  approximately  120,000  veterans  receiving 
disability  retired  pay).  Approximately  20,000  service  members  are  processed  in  the 
DBS  each  year,  in  addition  to  approximately  4,500  who  are  processed  from  the  TDRL. 
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Whereas  primary  participants  could  identify  specific  problems  with  the  management 
of  the  TDRL,  many  primary  participants  lacked  basic  information  regarding  the  ap¬ 
plication  of  disability  policy  to  Reserve  component  service  members.  This  lack  of 
basic  information  among  primary  participants  makes  it  impossible  to  assess  the 
consistency  of  application  of  disability  policy  and  overall  system  performance,  but  it 
suggests  that  neither  application  of  disability  policy  nor  system  performance  as  it 
applies  to  the  Reserve  component  is  up  to  the  DES  level  as  it  applies  to  the  active 
component. 

SUGGESTIONS  FOR  FUTURE  STUDY 

If  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and  the  ASD/RA,  decides  on  a 
purpose  statement  and  a  set  of  desired  outcomes  for  the  DES,  the  ASD/FMP  could 
direct  the  purpose-driven,  top-down  approach  described  in  Chapter  3  to  identify 
effective  interventions  in  the  aforementioned  two  areas;  (1)  management  of  the 
TDRL,  and  (2)  application  of  disability  policy  in  the  Reserve  components.  In  the 
absence  of  an  established  purpose  statement  and  set  of  desired  outcomes,  the 
Assistant  Secretaries  could  nevertheless  direct  the  issues-driven  approach  described 
in  Chapter  4  in  both  these  areas. 

In  any  event,  we  recommend  that  the  ASD/FMP,  in  coordination  with  the  ASD/HA 
and  the  ASD/RA,  address  TDRL  management  and  application  of  disability  policy  in 
the  Reserve  components. 

The  OSD  could  also  apply  the  two  approaches  used  in  this  report— issues-driven  and 
purpose-driven — to  identify  potential  problem  areas  and  develop  appropriate  inter¬ 
ventions  in  other  programs  in  which  consistent  application  of  defense-wide  policy  is 
an  important  factor,  or  to  improve  program  performance.  For  example,  potential 
candidates  that  might  benefit  from  the  approaches  described  in  this  report  include 
TRICARE  (the  military  health-care  program)  and  the  Defense  Leadership  and 
Management  Program  (the  DoD’s  training  program  for  senior  civilians). 

In  conclusion,  we  urge  the  Assistant  Secretary  of  Defense  for  Force  Management 
Policy,  in  coordination  with  the  Assistant  Secretary  of  Defense  for  Health  Affairs  and 
the  Assistant  Secretary  of  Defense  for  Reserve  Affairs,  to  decide  on  a  statement  of 
purpose  and  a  set  of  desired  outcomes  for  the  Disability  Evaluation  System  to  serve 
as  the  basis  for  developing  and  implementing  the  DoD  disability  evaluation  training 
program  and  a  supporting  management  information  system.  Developing  and  apply¬ 
ing  a  system-performance  perspective  to  the  DES  can  lead  the  way  to  improving 
overall  performance. 


Appendix  A 

OVERVIEW  OF  THE  OFFICE  OF  THE  UNDER  SECRETARY 
OF  DEFENSE  AND  ORGANIZATIONAL  RESPONSIBILITY 
FOR  THE  DISABILITY  EVALUATION  SYSTEM 


Department  of  Defense  Directive  1332.18  (1996)  and  Department  of  Defense 
Instruction  1332.38  (1996)  assign  organizational  responsibility  for  the  Disability 
Evaluation  System  in  the  Department  of  Defense  to  the  three  Assistant  Secretaries  of 
Defense— for  Force  Management  Policy,  Health  Affairs,  and  Reserve  Affairs— who 
report  to  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness,  and  to  the 
Secretaries  of  the  military  departments. 

Before  examining  the  specific  responsibilities  assigned  to  these  Assistant  Secretaries, 
a  brief  historical  overview  of  the  structure  of  the  Office  of  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness  is  instructive,  given  that  organizational  struc¬ 
ture  naturally  influences  organizational  behavior.  This  overview  establishes  the 
context  within  which  the  current  organizational  elements  interact  to  manage 
the  DES. 

An  Assistant  Secretary  of  Defense^  has  been  assigned  to  the  manpower  function 
since  1950.2  position  is  currently  titled  the  Assistant  Secretary  of  Defense  (Force 
Management  Policy)  Before  the  FY  1984  Defense  Authorization  Act  mandated  es¬ 
tablishment  of  an  additional  Assistant  Secretary  of  Defense  for  Reserve  Affairs,  the 
reserve  afl^airs  function  was  sometimes  combined  with  the  manpower  function, 
which  was  also  sometimes  combined  with  other  functions  under  the  original 
Assistant  Secretary  of  Defense  for  Manpower  position.  Likewise,  an  Assistant 
Secretary  of  Defense  has  been  assigned  to  the  health  affairs  function  since  1953,  ex¬ 
cept  from  1961  to  1970,  when  a  presidential  appointee  with  Senate  confirmation  was 
not  authorized  for  the  function. 

These  three  Assistant  Secretaries— for  Force  Management  Policy  (formerly  known  as 
Manpower),  Health  Affairs,  and  Reserve  Afl'airs — operated  with  relative  autonomy 
until  the  National  Defense  Authorization  Act  of  1994  created  the  new  Under 


^Assistant  Secretary  of  Defense  is  a  presidential  appointee  position  that  requires  Senate  confirmation. 

^The  DoD  was  established  in  1947,  making  the  Assistant  Secretary  of  Defense  one  of  the  longest  tenured 
positions  in  the  DoD. 

^See  Marcum  et  al.  (2001)  for  a  full  discussion  of  the  lineage  of  the  Assistant  Secretary  of  Defense  for 
Manpower  position. 
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Secretary  of  Defense  for  Personnel  and  Readiness.  Department  of  Defense  Directive 
5124.2  (1994)  delegated  authority  to  the  new  Under  Secretary  of  Defense  for 
Personnel  and  Readiness  to  exercise  authority,  direction,  and  control  over  the  three 
Assistant  Secretaries. 

Department  of  Defense  Directive  1332.18  and  Department  of  Defense  Instruction 
1332.38  assign  to  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness,  the 
Assistant  Secretaries  of  Defense,  and  the  Secretaries  of  the  military  departments  the 
following  responsibilities. 

The  Under  Secretary  of  Defense  (Personnel  and  Readiness),  under  DoD  Instruction 
1332.38 

•  exercises  cognizance  and  oversight  of  the  DoD  DBS 

•  makes  the  final  decision  on  requests  from  the  military  departments  for  excep¬ 
tions  to  the  standards  in  [DoDI  1332.38]. 

The  Assistant  Secretary  of  Defense  (Force  Management  Policy),  under  DoDD  1332.18 

•  develops  and  maintains,  in  coordination  with  the  Assistant  Secretary  of  Defense 
for  Health  Affairs  and  the  Assistant  Secretary  of  Defense  for  Reserve  Affairs,  a 
program  of  instruction  for  the  DBS 

•  monitors  changes  and  proposed  changes  to  military  personnel  and  compensa¬ 
tion  statutes  and  DoD  policy,  and  other  pertinent  authorities,  to  assess  their  im¬ 
pact  on  physical  disability  evaluation.  Reserve  component  medical  disqualifica¬ 
tion,  and  related  benefits;  and  issues  timely  guidance  to  the  military  services,  as 
appropriate 

•  coordinates  with  the  Assistant  Secretary  of  Defense  for  Health  Affairs  and  the 
Assistant  Secretary  of  Defense  for  Reserve  Affairs  in  developing  policy  for  referral 
of  members  into  the  DBS 

•  issues  and  maintains  DoD  Instruction  1332.38. 

The  Assistant  Secretary  of  Defense  (Force  Management  Policy),  under  DoD 
Instruction  1332.38 

•  exercises  cognizance  of  laws,  policies,  and  regulations  that  affect  the  DBS 

•  issues  guidance,  as  required,  to  further  interpret,  implement,  and  govern  the  pol¬ 
icy  and  procedures  for  the  four  elements  of  the  DBS 

•  establishes  necessary  reporting  requirements  to  monitor  and  assess  the  perfor¬ 
mance  of  the  DBS  and  compliance  of  the  Military  Departments  with  [DoDI 
1332.38]  and  DoDD  1332.18 

•  coordinates  with  the  Assistant  Secretary  of  Defense  for  Reserve  Affairs  concern¬ 
ing  the  impact  of  laws  and  DoD  policy  on  Reserve  members  who  have  conditions 
that  are  cause  for  medical  disqualification 
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•  coordinates  with  the  Assistant  Secretary  of  Health  Affairs  in  developing  proce¬ 
dures  for  medical  issues  pertaining  to  physical  disability  evaluation 

•  reviews  substantive  changes  proposed  by  the  military  departments  to 
departmental  policies  and  procedures  for  physical  disability  evaluation  that  af¬ 
fect  the  uniformity  of  standards  for  separation  or  retirement  for  unfitness  be¬ 
cause  of  physical  disability  or  separation  of  Ready  Reserve  members  for  medical 
disqualification 

•  develops  quality-assurance  procedures  to  ensure  that  policies  are  applied  in  a 
fair  and  consistent  manner. 

The  Assistant  Secretary  of  Defense  (Health  Affairs),  under  DoD  Directive  1332.18 

•  monitors  changes  to  the  statutes,  laws,  and  regulations  of  the  Department  of 
Veterans  Affairs  to  assess  their  impact  on  the  Department  of  Defense’s  applica¬ 
tion  of  the  Veterans  Administration  Schedule  for  Rating  Disabilities  (VASRD)  to 
service  members  determined  unfit  because  of  physical  disability,  and  issues 
timely  guidance  to  the  Military  Services,  as  appropriate,  upon  coordination  with 
the  Assistant  Secretary  of  Defense  for  Force  Management  Policy 

•  develops  and  periodically  reviews  medical  standards  for  referral  of  service  mem¬ 
bers  into  the  DES 

•  recommends  changes  to  and  maintains  DoD  Instruction  1332.39 

•  monitors  the  medical  element  of  the  DES  and  proposes  corrective  actions  as  re¬ 
quired 

•  develops  policies  for  the  medical  component  of  the  DES,  to  include  the  estab¬ 
lishment  of  minimum  standards  for  Medical  Evaluation  Boards,  Reserve 
component  medical  examinations  forwarded  to  Physical  Evaluation  Boards,  and 
TDRL  periodic  examinations 

•  develops  and  maintains  a  program  of  instruction  for  use  by  MTFs  on  the 
preparation  of  Medical  Evaluation  Boards  for  physical  disability  cases 

•  develops  a  program  of  instruction  for  use  by  PEB  adjudicators  and  appellate  re¬ 
view  authorities  on  the  medical  aspects  of  physical  disability  adjudication,  to  in¬ 
clude  the  application  of  the  VASRD 

•  monitors  the  timeliness  of  the  medical  component  of  the  DES 

•  develops  policy  for  conduct  of  maximum  interval  physical  examinations  and 
certification  of  physical  condition  for  members  of  the  Reserve  components. 

The  Assistant  Secretary  of  Defense  (Health  Affairs),  under  DoD  Instruction  1332.38 

•  makes  recommendations  for  a  final  decision  by  the  Secretary  of  Defense  on  the 
unfit  findings  on  all  officers  in  pay  grade  0-7  or  higher  and  medical  officers  in 
any  grade  who  are  pending  nondisability  retirement  for  age  or  length  of  service 
at  the  time  of  their  referral  into  the  DES 
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•  reviews  substantive  changes  proposed  by  the  military  departments  in  their  sup- 
plemental  medical  standards  to  enclosure  4  of  [DoDI  1332.38]  concerning  medi¬ 
cal  conditions  that  are  cause  for  referral  of  a  member  into  the  DES. 

The  Assistant  Secretary  of  Defense  (Reserve  Affairs),  under  DoD  Directive  1332.18 

•  ensures  the  policies  for  the  DES  are  applicable  to  members  of  the  Ready  Reserve 
and  those  policies  for  the  Ready  Reserve  are  consistent  with  the  policies  estab¬ 
lished  for  active  component  personnel. 

The  Assistant  Secretary  of  Defense  (Reserve  Affairs),  under  DoD  Instruction  1332.38 

•  Coordinates  as  necessary  to  ensure  that  procedures  for  the  DES  apply  consis¬ 
tently  and  uniformly  to  members  of  the  Reserve  components. 

The  Secretaries  of  the  military  departments,  under  DoD  Instruction  1332.18 

•  ensure  compliance  with  Chapter  61  of  10  U.S.C.,  [DoD  Directive  1332.18],  and 
instructions  and  guidance  issued  under  [its]  authority 

•  establish  the  service-specific  DES  to  consist  of  the  four  components:  medical 
evaluation,  physical  disability  evaluation,  to  include  appellate  review;  counsel¬ 
ing:  and  final  disposition 

•  manage  the  service-specific  DES  to  ensure  physical  disability  evaluation  is  ac¬ 
complished  in  a  timely  manner  with  uniform  application  of  the  governing  laws 
and  DoD  policy 

•  ensure  that  physicians  who  serve  on  Medical  Evaluation  Boards  are  trained  in 
the  preparation  of  medical  boards  for  physical  disability  evaluation 

•  ensure  that  PEB  members  and  applicable  review  authorities  are  trained  and 
certified  in  physical  disability  evaluation 

•  ensure  all  matters  raising  issues  of  fraud  within  the  DES  are  investigated  and  re¬ 
solved  as  appropriate 

•  defer  a  determination  of  disability  retirement  of  any  officer  who  is  being  pro¬ 
cessed  for,  is  scheduled  for,  or  has  received  nondisability  retirement  for  age  or 
length  of  service  until  such  determination  is  approved  by  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness  on  the  recommendation  of  the  Assistant 
Secretary  of  Defense  for  Health  Affairs  under  Section  1216(b)  of  Title  10,  U.S.C. 

The  Secretaries  of  the  Military  Departments,  under  DoD  Instruction  1332.38 

•  ensure  that  members  with  conditions  that  may  be  cause  for  referral  into  the  DES 
are  counseled  at  appropriate  stages  on  the  DES  process  and  the  member’s  rights, 
entitlements,  and  benefits 

establish  a  quality-assurance  process  to  ensure  that  policies  and  procedures  es¬ 
tablished  by  DoDD  1332.18  and  [DoDI  1332.38]  are  interpreted  uniformly 
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make  determinations  on  unfitness  because  of  medical  disqualification  or  physi- 
cal  disability;  entitlement  to  assignment  of  percentage  of  disability  at  the  time  of 
retirement  or  separation  because  of  physical  disability;  and  except  as  limited  by 
10  U.S.C.  1216(d),  entitlement  to  and  payment  of  disability  retired  and  severance 
pay 

ensure  that  the  record  of  proceedings  for  physical  disability  cases  supports  the 
findings  and  recommendations  made 

ensure  the  Temporary  Disability  Retired  List  is  managed  to  meet  the  require¬ 
ments  of  10  U.S.C.  1210  for  timely  periodic  physical  examinations,  suspension  of 
retired  pay,  and  removal  from  the  TDRL 

designate  a  military  department  representative  to  serve  as  the  department  rep¬ 
resentative  for  the  Disability  Evaluation  System 

ensure  all  matters  raising  issues  of  fraud  on  the  DES  by  members  are  investigated 
and  resolved  as  appropriate. 


Appendix  B 


MILITARY  DEPARTMENT  INTERPRETATIONS  OF  THE  PURPOSE 

OF  THE  DISABILITY  EVALUATION  SYSTEM 


No  shared  philosophy  within  the  OSD  or  across  military  departments  defines  the 
fundamental  reason  why  the  Disability  Evaluation  System  exists— that  is,  there  is  no 
shared  statement  of  its  purpose.  Lacking  direction  from  the  OSD  on  the  purpose  of 
the  DES,  the  military  departments  fill  the  void  by  giving  DoD  policy  language  their 
own  interpretation  and  operate  their  systems  accordingly. 

The  official  documents  that  govern  the  military  departments’  systems  express  the 
purpose  or  objectives  of  their  respective  DES  differently.  Army  Regulations  govern 
the  operations  of  the  Army  DES,  Secretary  of  the  Navy  Instructions  govern  the  op¬ 
erations  of  the  Department  of  the  Navy  DES,  and  Air  Force  Instructions  govern  the 
operations  of  the  Air  Force  DES. 


DEPARTMENT  OF  THE  ARMY 

The  Army  describes  the  purpose  of  its  governing  regulation  (AR  635-40,  para.  I-l)  as 

being  designed  to 

•  maintain  an  effective  and  fit  military  organization  with  maximum  use  of  avail¬ 
able  manpower 

•  provide  benefits  for  eligible  soldiers  whose  military  service  is  terminated  because 
of  a  service-connected  disability 

•  provide  prompt  disability  processing  while  ensuring  that  the  rights  and  interests 
of  the  United  States  government  and  the  soldier  (service  member)  are  protected. 

DEPARTMENT  OF  THE  NAVY 

The  Department  of  the  Navy  governing  document  (Secretary  of  the  Navy  Instruction 

1850.4D,  1998,  para.  3101)  focuses  on  objectives,  as  follows: 

•  The  maintenance  of  a  physically  fit  and  combat-ready  Navy  and  Marine  Corps, 
including  Reserve  components 

•  Equitable  consideration  of  the  interests  of  the  government  and  individual  service 
members. 
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DEPARTMENT  OF  THE  AIR  FORCE 

The  Air  Force  governing  document  (AFI  36-3212,  1998.  p.  10)  describes  the  purpose 
of  its  Disability  Evaluation  System  as  follows: 

To  maintain  a  fit  and  vital  force,  disability  law  allows  the  Secretary  of  the  Air  Force  to 
remove  from  active  duty  those  members  who  can  no  longer  perform  the  duties  of 
their  office,  grade,  rank,  or  rating  and  ensure  fair  compensation  to  members  whose 
military  careers  are  cut  short  due  to  a  service-incurred  or  service-aggravated  physical 
disability. 


JOINT  SERVICE  DISABILITY  WORKING  GROUP 

To  address  the  findings  of  a  DoD  Inspector  General  audit  report  dated  June  1992,  the 
Office  of  the  ASD/HA  convened  a  Joint  Service  Disability  Working  Group  to  analyze 
the  disability  evaluation  process  using  the  Corporate  Information  Management 
methodology.  That  group  articulated  the  following  mission  statement  for  the  DOD 
Disability  Evaluation  System: 


The  military  disability  evaluation  process  provides  and  maintains  a  fit  force,  removes 
unfit  members  from  active  duty  who  can  no  longer  perform  duties  commensurate 
with  their  office,  grade,  rank,  or  rating,  and  provides  compensation  to  members 
whose  military  careers  are  cut  short  due  to  a  service-incurred  or  service-aggravated 
physical  disability  (Joint  Service  Disability  Working  Group,  November  17, 1963). 

SUMMARY 

The  four  statements  of  the  DES  purpose,  objectives,  or  mission  in  this  appendix— all 

seeking  to  establish  the  reason  the  system  exists — offer  various  renditions  and 

combinations  of  four  themes: 

1.  All  four  statements  include  a  “maintain  a  fit  force”  theme,  as  follows: 

•  Maintain  an  effective  and  fit  military  organization  with  maximum  use  of  avail¬ 
able  manpower. 

•  Maintain  a  physically  fit  and  combat-ready  Navy  and  Marine  Corps. 

•  Maintain  a  fit  and  vital  force. 

•  Provide  and  maintain  a  fit  force. 

2.  Three  statements  include  a  “provide  compensation  and  benefits”  theme; 

•  Provide  benefits  for  eligible  soldiers  whose  military  service  is  terminated  because 
of  a  service-connected  disability. 

•  Ensure  fair  compensation  to  members  whose  military  careers  are  cut  short  due 
to  a  service-incurred  or  service-aggravated  physical  disability. 

•  Provide  compensation  to  members  whose  military  careers  are  cut  short  due  to  a 
service-incurred  or  service-aggravated  physical  disability. 
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3.  Two  statements  include  a  “remove  unfit  members  from  active  duty”  theme: 

•  Provide  prompt  disability  processing;  remove  from  active  duty  those  who  can  no 
longer  perform  the  duties  of  their  office,  grade,  rank,  or  rating. 

•  Remove  unfit  members  from  active  duty  who  can  no  longer  perform  duties 
commensurate  with  their  office,  grade,  rank,  or  rating. 

4.  Two  statements  include  a  “balance  the  interests  of  the  government  and  the 
service  member”  theme: 

•  Ensure  that  the  rights  and  interests  of  the  government  and  the  soldier  are 
protected. 

•  Ensure  equitable  consideration  of  the  interests  of  the  government  and  individual 
service  members. 


Appendix  C 

DISABILITY  EVALUATION  SYSTEM  ISSUES 


This  appendix  contains  a  comprehensive  list  of  issues  associated  with  the  Disability 
Evaluation  System.  We  identified  these  issues — ^which  address  instances  of  variability 
in  policy  application  across  or  within  the  military  departments,  as  well  as  some  spe¬ 
cific  problems  associated  with  the  DBS — during  interviews  with  numerous  and  di¬ 
verse  primary  participants,  and  in  the  course  of  attending  the  military  departments’ 
major  training  events.  These  issues  serve  as  the  basis  for  the  goal  fabric  analysis  and 
the  resulting  actions  in  the  ten  categories  of  interventions  discussed  in  Chapter  4. 

1.  Medical  Evaluation  Boards  convened  too  early  (for  example,  shortly  before 
surgery  or  immediately  after  post-injury/illness  period). 

2.  The  services  employ  different  philosophies  for  referring  service  members  for 
Medical  Evaluation  Boards.  For  example,  the  majority  of  Army  service  members 
who  receive  Medical  Evaluation  Boards  are  referred  to  the  PEB  and  the  majority 
of  those  are  found  unfit,  whereas  the  majority  of  Air  Force  service  members  who 
receive  Medical  Evaluation  Boards  have  a  high  probability  of  returning  to  duty. 

3.  DoDD  1332.18  holds  the  Secretaries  of  the  military  departments  responsible  for 
ensuring  that  physicians  who  serve  on  Medical  Evaluation  Boards  are  trained  in 
the  preparation  of  medical  boards  for  physical  disability  evaluation.  No  institu¬ 
tional  mechanism  exits  in  any  of  the  military  departments  to  do  this.  The 
Departments  of  the  Navy  and  the  Air  Force  claim  to  be  in  the  process  of 
updating  instructions  (published  official  governing  documents)  that  describe 
how  to  conduct  Medical  Evaluation  Boards. 

4.  In  the  Department  of  the  Navy,  the  Medical  Evaluation  Board  fairly  frequently 
(in  5  to  10  percent  of  the  cases)  refers  medical  boards  that  do  not  qualify  for  the 
DES  to  the  PEB  because  the  referring  physicians  and  commanders  do  not 
communicate  with  each  other  and  the  physicians  play  a  strong  patient  advocate 
role. 

5.  No  written  retention  standards  exist  (except  for  the  Army)  and  the  services  use 
different  retention  standards.  For  example,  the  Army  refers  service  members 
with  asthma  to  the  Medical  Evaluation  Boards. 

6.  Confusion  exists  among  members  of  the  Disability  Advisory  Council  and  mem¬ 
bers  of  the  PEBs  regarding  reasons  for  nondeployability  and  use  of  nondeploy¬ 
ability  in  determinations  of  fitness. 
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7.  Different  military  departments’  informal  PEBs  receive  different  information 
upon  which  to  make  judgments. 

8.  The  military  departments  allow  service  members  different  lengths  of  time  (3, 10, 
or  15  days)  to  make  an  election  of  informal  PEB  and  formal  PEB  findings,  which 
impacts  service  members’  perceptions  of  due  process. 

9.  No  reliable  information  system  exists  to  present  performance  data  to  MTF  com¬ 
manders  and  the  Surgeons  Generals. 

10.  Some  medical  boards  are  not  processed  in  a  timely  manner;  they  linger  in  the 
system  and  are  then  referred  to  the  PEB  after  the  narrative  summary  and/or  the 
specialty  consults  are  more  than  90  days  old. 

11.  DoD  Directive  1332.18  holds  Secretaries  of  the  military  departments  responsible 
for  ensuring  that  physicians  who  serve  on  Medical  Evaluation  Boards  are 
trained  in  the  preparation  of  medical  boards  for  physical  disability  evaluation; 
however,  no  institutional  mechanism  exists  in  any  of  the  military  departments 
to  ensure  that  this  happens.  Across  the  military  departments,  doctors  typically 
receive  no  standardized  training  in  writing  medical  boards  (narrative 
summaries)  or  specialty  consults.  Some  new  doctors  may  receive  a  “crash 
course’’  on  writing  narrative  summaries,  but  nothing  standardized  or 
consistently  used  exists  within  or  across  the  military  departments. 

12.  Many  doctors  have  no  understanding  of  the  DES,  or  they  lack  knowledge  about 
the  information  the  PEB  needs  to  make  appropriate  assessments.  Many  do  not 
fully  understand  some  basic  concepts  of  the  DES,  such  as  “service  aggravation,” 
“presumption  of  fitness,”  or  “fit/unfit.” 

13.  An  adversarial  relationship  exists  between  referring  physicians  and  the 
Department  of  the  Navy  PEB.  Doctors  spend  16  months  (Limited  Duty 
maximum  time)  treating  a  service  member’s  medical  condition  and  when  they 
cannot  resolve  the  condition  during  that  time,  they  refer  the  service  member  to 
the  PEB  with  the  expectation  that  the  PEB  will  find  the  member  “unfit.”  In  cases 
such  as  these,  some  doctors  tend  to  regard  a  “fit”  call  by  the  PEB  as  a  personal 
affront  to  their  medical  expertise.  Referring  physicians  oftentimes  do  not 
understand  that  a  PEB  determination  of  “fit  for  duty”  is  not  equivalent  to  “fit  for 
full  deployment.” 

14.  Nondeployable  service  members  are  a  particular  problem  in  the  Department  of 
the  Navy  because  the  Navy  does  not  have  many  shore  billets.  Most  Department 
of  the  Navy  assignments  require  four  years  of  shore  duty,  then  three  years  of  sea 
duty.  Because  of  this  rotation  policy,  some  line  officers,  physicians,  medical 
policymakers,  and  assignments  personnel  in  the  Department  of  the  Navy  would 
like  to  see  the  fit  call  and  suitability  standards  more  closely  aligned.  Likewise, 
some  operational  leaders,  physical  disability  evaluation  policymakers,  and 
assignments  personnel  in  the  Army  also  would  like  to  see  the  deployability  and 
fit  calls  more  closely  aligned. 

15.  The  Department  of  the  Navy  PEB  is  finding  an  increasing  number  of  service 
members  fit.  The  fit  calls  have  roughly  doubled  over  the  past  few  years;  about 
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30  percent  of  the  PEB  adjudications  result  in  fit  calls.  Is  the  increase  caused  by  a 
change  in  the  PEB  philosophy,  or  a  change  in  the  quality  of  the  Medical 
Evaluation  Board  or  Physical  Evaluation  Board?  The  PEBs  first  got  access  to  the 
commanders’  nonmilitary  assessment  following  publication  of  the  new 
Secretary  of  the  Navy  Instruction  1850.4D  in  December  1998,  and  the  fit  calls 
subsequently  went  up  (reportedly  because  the  PEBs  now  had  access  to  the  com¬ 
mander’s  input). 

16.  DoD  Directive  1332.18  and  the  new  SECNAVINST  1850.4D  contain  examples  of 
a  good  medical  board.  Copies  reportedly  are  rarely  made  available  to  doctors, 
and  most  doctors  do  not  use  them  even  when  they  are  available. 

17.  Physicians  across  the  military  departments  who  write  specisdty  consults  (from 
orthopedics,  pulmonary,  cardiac,  and  other  specialty  areas)  need  to  be  informed 
about  the  five  specific  points  they  must  address  when  writing  their  consults  in 
order  for  the  PEBs  to  adjudicate  the  cases  (for  example,  sufficiently  document¬ 
ing  specific  range-of-motion  ratings).  Cardiac  cases  most  often  lack  the  suffi¬ 
ciently  detailed  information. 

18.  Commanders  typically  do  not  understand  the  role  or  purpose  of  the  DES,  or 
their  role  within  the  system.  In  particular,  they  do  not  seem  to  understand  the 
“tail”  (that  is,  the  vast  amount  of  resources)  that  follows  referrals  to  the  Medical 
Evaluation  Board. 

19.  In  the  Army,  no  one  physician  is  responsible  for  moving  a  patient’s  case  through 
the  health-care  system  to  a  Medical  Evaluation  Board.  As  a  result,  the  patient 
gets  passed  on  for  specialty  consults  and  the  case  can  get  lost  in  the  process.  In 
the  Air  Force,  the  initial  contact  physician  for  the  specialty  ensures  that  the 
appropriate  consults  are  done,  the  narrative  summary  is  dictated,  and  the 
medical  board  is  consistent  and  complete,  and  the  medical  board  together  with 
the  outpatient  records  is  delivered  to  the  Medical  Evaluation  Board.  In  the  Air 
Force,  the  MTF  commander  ensures  that  the  attending  physician  notifies  the 
PEBLOs  as  soon  as  it  appears  likely  that  a  service  member  will  require 
evaluation  to  determine  physical  fitness  for  retention  in  a  duty  status  (U.S. 
Department  of  the  Air  Force,  Physical  Disability  Division,  1999). 

20.  Most  Department  of  the  Navy  service  members  are  referred  to  Medical 
Evaluation  Boards  by  the  Limited  Duty  Boards  because  the  service  member  has 
spent  too  much  time  in  a  “limited  duty”  status.^ 

21.  PEBs  return  insufficient  or  incomplete  medical  boards  to  the  referring  MTF  for 
the  following  reasons;  no,  or  insufficient,  nonmedical  assessment  or  comman¬ 
der’s  letter;  missing  LODD;  missing  appropriate  specialty  consult(s)  or  the 
consults  lack  sufficient  detail  to  adjudicate  a  case,  for  example,  no  social  and 


^In  the  Department  of  the  Navy,  a  service  member  may  receive  up  to  30  days  of  "light  duty”  while 
undergoing  treatment  for  a  medically  diagnosed  condition.  If  the  member  continues  to  need  medical 
treatment  at  the  end  of  the  30-day  period,  he  or  she  may  be  referred  to  a  Limited  Duty  Board  or  a  Medical 
Evaluation  Board  for  further  evaluation.  The  Limited  Duty  Board  may  grant  the  member  up  to  16  months 
of  Temporary  Limited  Duty  (in  up  to  eight-month  increments)  and  the  member  may  spend  up  to  30  days 
in  Medical  Hold  pending  completion  of  a  Medical  Evaluation  Board  referral  to  the  PEB. 
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industrial  impairment  assessment  on  psychiatric  cases;  insufficient  signatures 
(number  and  specialty  of  physicians,  and  the  approving  authority  must  be  a 
medical  officer);  the  narrative  summary  is  more  than  90  days  old  and  therefore 
no  longer  reliable;  incomplete  narrative  summaries  that  lack  the  level  of  detail 
needed  to  make  a  fit/unfit  determination  and  apply  the  VASRD. 

22.  PEBLOs  with  greater  tenure,  program  knowledge,  and  experience  may  be  (1) 
more  likely  to  provide  service  members  with  accurate  expectations  of  the  DES 
because  they  can  better  explain  the  VASRD  rating  and  disability  compensation; 
(2)  more  effective  in  soliciting  commanders’  letters  and  LODDs;  and  (3)  more 
comfortable  and  effective  in  using  the  chain  of  command  to  solve  problems 
such  as  unit  commander  nonresponsiveness. 

23.  The  members  of  the  Disability  Advisory  Council  and  military  department’s  pri- 
mary  participants  communicate  with  one  another  infrequently. 

24.  Although  DoD  Directive  1332.18  identifies  medical  evaluation  as  one  of  four  ele¬ 
ments  of  the  DES,  the  medical  community  does  not  seem  to  accept  ownership 
of  this  element  given  its  apparent  reluctance  to  (1)  respond  to  requests  from  the 
PEB  to  train  physicians  who  write  narrative  summaries  and  specialty  consults  to 
meet  certain  standards;  (2)  incorporate  its  governing  documents  into  the  overall 
DES  documents;  (3)  strike  the  appropriate  balance  in  terms  of  advocacy— physi¬ 
cians  seem  to  emphasize  their  role  of  service  member  advocate  over  their 
equally  important  role  of  military  department  advocate. 

25.  Data  with  which  to  make  an  assessment  are  not  generally  provided  to  senior  of¬ 
ficials  charged  with  quality  assurance. 

26.  Secretaries  of  the  military  departments  receive  no  information  regarding  how 
well  the  DES  is  working. 

27.  Human  elements— such  as  emotions,  personality  issues,  good  soldier/bad  sol¬ 
dier  issues,  and  length  of  service  (when  close  to  20  years)— hamper  efforts  to 
render  fair  and  consistent  decisions.  PEB  decisions  change  with  new  members’ 
personal  philosophies. 

28.  The  Department  of  the  Navy  reserves  “Permanent  Limited  Duty”  status  for 
members  who  have  a  “significant  number  of  years  in  service  and  want  to  retire,” 
who  are  very  close  to  retirement  when  found  unfit,  or  who  have  special 
expertise.  This  tendency  seems  to  be  consistent  with  DoD  Directive 
1332.18.3.12:  “As  an  exception  to  general  policy,  the  Secretary  concerned,  upon 
the  request  of  the  member  or  upon  the  exercise  of  discretion  based  on  the  needs 
of  the  Service,  may  continue  in  a  permanent  limited  duty  status  either  on  active 
duty  or  in  the  Ready  Reserve,  a  member  determined  unfit  because  of  physical 
disability  when  the  member’s  Service  obligation  or  special  skill  and  experience 
justifies  such  continuation.” 

29.  Primary  participants  interpret  DoD  policy  and  apply  it  consistently  to  the  best  of 
their  ability.  However,  primary  participants  do  not  converse  with  their  counter¬ 
parts  from  the  other  military  departments  or  the  OSD,  so  they  have  no  way  of 


Disability  Evaluation  System  Issues  165 


knowing  if  they  are  passing  judgments  that  differ  from  those  of  their  counter¬ 
parts  in  the  other  military  departments. 

30.  The  Departments  of  the  Navy  and  Army  PEBs  are  physically  located  in  different 
regions  of  the  country  and  rarely  communicate  with  one  another. 

31 .  DoD  Directive  1332. 18  holds  the  Secretaries  of  the  military  departments  respon¬ 
sible  for  ensuring  PEB  members  and  applicable  review  authorities  are  trained 
and  certified  in  physical  disability  evaluation.  No  institutional  mechanism  exists 
in  any  of  the  military  departments  to  ensure  that  PEB  members  emd  applicable 
review  authorities  are  “certified”  in  physical  disability  evaluation. 

32.  Physicians  often  write  medical  boards  for  service  members  firom  other  military 
departments.  The  different  military  departments  write  medical  boards  in  differ¬ 
ent  ways.  Some  primary  participants  think  that  the  ASD/HA  should  require  a 
standard  format  for  medical  boards  in  all  of  the  military  departments. 

33.  0-6s  who  serve  as  PEB  president  in  the  Department  of  the  Navy  typically  stay  in 
the  position  for  only  about  six  months.  This  turnover,  or  lack  of  continuity,  pre¬ 
cludes  those  leaders  from  developing  real  commitment  to  the  PEB  mission  and 
it  is  nearly  impossible  for  them  to  champion  needed  change,  such  as 
streamlining  operating  procedures  and  revising  policy  documents,  let  alone  the 
more  difficult  challenge  of  changing  the  PEB  philosophy. 

34.  Many  primary  participants  suggested  that  senior  OSD  leadership  appears  to 
take  very  little  interest  in  the  DES  and  exercises  practically  no  authority  in  assur¬ 
ing  that  it  operates  as  it  should.  They  perceive  that  the  only  real  DES  oversight 
comes  firom  the  Director  of  Officer  and  Enlisted  Personnel  Management. 

35.  No  DES  process  owner  exists;  none  of  the  primary  participants  (except  the 
PEBLOs  in  the  Department  of  the  Navy  medical  centers)  in  the  medical  evalua¬ 
tion  phase  of  the  DES  work  for  the  0-6  who  oversees  the  PEB,  who  is  also  the 
PEB  approving  authority. 

36.  Service  member  patients  perceive  variability  in  the  application  of  disability  pol¬ 
icy  because  both  the  DoD  and  VA  rate  a  physician’s  diagnosis  using  the  VASRD, 
but  DoD  and  VA  actually  evaluate  different  things  for  different  purposes  at 
different  times. 

37.  Navy  PEBLO  positions  are  filled  with  service  members  from  a  variety  of  career 
fields  with  skill  sets  that  are  far  removed  from  patient  administration  and  coun¬ 
seling. 

38.  Military  departments  describe  the  purpose  of  DES  differently. 

39.  Referrals  to  pre-separation  counseling  before  a  PEB  decision  of  unfitness  may 
lead  to  incorrect  and  premature  expectations  of  the  service  member. 

40.  Military  departments  conduct  Medical  Evaluation  Boards  in  different  forms 
(convene  or  pass  medical  boards). 

41.  We  observed  junior  noncommissioned  officers  and  petty  officers  acting  as 
PEBLOs  and  Department  of  the  Navy  disability  evaluation  counselors. 
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42.  No  institutional  mechanism  exists  for  quality  assurance. 

43.  Fitness  and  rating  calls  for  members  with  close  to  but  less  than  20  years  of 
service  are  inconsistent  because  of  the  desire  of  some  services  and  some  PEBs  to 
retire  such  individuals. 


Appendix  D 

EXAMPLE  OF  GOAL  FABRIC  ANALYSIS  DEVELOPMENT 


This  appendix  illustrates  the  goal  fabric  analysis  framework  described  in  Chapter  4.  It 
presents  a  single  example,  starting  with  three  related  issues  (see  Appendix  C  for  the 
complete  list  of  DES  issues),  and  identifies  two  desired  results  associated  with  the  is¬ 
sues,  five  actions  that  support  the  desired  results,  and  one  objective  that  the  desired 
results  suggest.  This  appendix  then  presents  the  full  set  of  objectives  and  the  goals 
they  support,  based  on  this  issues-based,  bottom-up  analytic  approach.  We  also  il¬ 
lustrate  the  relationships  between  issues,  desired  results,  actions,  actual  results,  ob¬ 
jectives,  and  goals. 

The  illustrative  example  begins  with  the  following  three  related  issues: 

•  The  military  departments  adhere  to  different  philosophies  when  referring  service 
members  for  Medical  Evaluation  Boards. 

•  In  the  Department  of  the  Navy,  the  Medical  Evaluation  Board  fairly  frequently  (in 
5  to  10  percent  of  the  cases)  refers  medical  boards  that  do  not  qualify  for  the  DES 
to  the  Informal  PEB  because  the  referring  physicians  and  commanders  do  not 
communicate  with  each  other  and  the  physicians  play  such  a  strong  patient  ad¬ 
vocate  role. 

•  Service  member  patients  perceive  variability  in  application  of  disability  policy 
because  both  the  DoD  and  VA  rate  the  physician’s  diagnosis  using  the  VASRD, 
but  each  department  actually  evaluates  different  things  for  different  purposes  at 
different  times. 

The  first  two  issues  listed  here  suggest  the  following  desired  result:  Physicians  under¬ 
stand  the  purpose  and  role  of  the  Medical  Evaluation  Board  in  the  overall  Disability 
Evaluation  System  (the  purpose  and  role  could  differ  among  the  services).  The  third 
issue  suggests  the  following  desired  result:  Service  members  understand  the  different 
purposes  of  DoD  and  VA  disability  evaluation.  Figure  D.l  portrays  the  relationships 
between  the  issues  and  results. 

After  examining  the  complete  list  of  41  issues  (see  Appendix  C)  and  their  associated 
desired  results  using  the  bottom-up,  issues-based  goal  fabric  analytic  technique,  we 
conjectured  an  initial  set  of  objectives  that  the  group  of  desired  results  seemed  to 
suggest.  For  the  sake  of  illustration,  the  two  desired  results  shown  in  Figure  D.l  seem 
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to  suggest  the  following  objective:  Communicate  the  purpose  and  role  of  the 
Disability  Evaluation  System. 

For  both  of  the  desired  results  shown  in  Figure  D.l,  we  also  specified  actions  that 
could  bring  about  those  results.  For  example,  to  ensure  that  physicians  understand 
the  purpose  and  role  of  the  Medical  Evaluation  Board  in  the  overall  Disability 
Evaluation  System  (the  purpose  and  role  could  differ  among  the  services),  we  suggest 
two  actions:  In  coordination  with  the  ASD/RA  and  the  ASD/HA,  the  ASD/FMP  should 
direct  the  Director  of  Officer  and  Enlisted  Personnel  Management  to  (1)  consult  with 
a  small  group  of  experienced  DES  experts  representing  the  military  departments’ 
PEBs  and  the  Office  of  the  Surgeons  General  to  produce  recommendations  upon 
which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and  ASD/RA,  can 
decide  on  a  purpose  and  role  of  the  Medical  Evaluation  Board  and  (2)  develop  and 
promulgate  the  clearly  stated  purpose  and  role  of  the  Medical  Evaluation  Board 
within  the  overall  DES. 

For  the  second  desired  result— service  members  understand  the  different  purposes 
of  DoD  and  VA  disability  evaluations— the  suggested  actions  include  (1)  the  OSD 
develops  a  brochure  and/or  Web  site  for  individuals  who  are  separated  or  retired  for 
disability  that  focuses  on  the  differences  between  the  DoD  and  VA  systems:  services 
would  present  the  brochure  and/or  Web  site  to  members  during  outprocessing  at  the 
transition  points;  (2)  the  OSD  develops  a  PEBLO  and/or  service  member  Web  site 
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Figure  D.l — Issues  Leading  to  Desired  Results 


Example  of  Goal  Fabric  Analysis  Development  169 


and  lists  the  differences  between  the  DoD  and  the  VA  disability  systems  under 
Frequently  Asked  Questions;  (3)  the  OSD  includes  the  differences  between  the  DoD 
and  VA  systems  as  a  just-in-time  training  and  information  topic  (on  a  CD-ROM  or 
Web  site);  and  (4)  the  OSD  makes  understanding  of  DoD  and  VA  systems  part  of  the 
PEBLO  certification  process. 

All  the  individual  parts— desired  results  and  actions— fit  together  and  become  oper¬ 
ational  when  “actual  results”  are  added  to  the  picture  (see  Figure  D.2).  Desired  re¬ 
sults  suggest  both  objectives  and  actions  to  achieve  them;  when  the  actions  are  car¬ 
ried  out,  actual  results  are  obtained.  If  the  actual  results  when  compared  with  the 
desired  results  are  consistent  with  the  desired  results,  those  actions  clearly  con¬ 
tribute  to  achievement  of  the  stated  objective.  If  they  are  not  consistent  with  the  de¬ 
sired  results,  the  actions  (or  action)  should  be  modified  to  produce  actual  results  that 
are  consistent  with  desired  results.  Figure  D.2  portrays  the  relationships  between  ac¬ 
tions  and  results  in  light  of  the  stated  system  objective  “Communicate  purpose  and 
role  of  Disability  Evaluation  System.” 
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Figure  D.2— Relationship  Between  Actions  and  Desired  and  Actual  Results 
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The  41  issues  identified  during  the  goal  fabric  analysis  led  to  a  variety  of  desired 
results,  actions,  and  objectives.  The  relationships  among  them  became  quite 
complex.  Figure  D.2  portrays  the  kinds  of  complex  relationships  that  build  up  to 
goals.  In  addition.  Figure  D.3  illustrates  how  multiple  desired  results  can  serve  one 
objective,  and  how  more  than  one  objective  can  support  a  single  goal.  Each  issue 
suggests  one  desired  result;  also,  several  issues  may  suggest  the  same  desired  result. 
Each  desired  result  can  suggest  multiple  actions  and,  although  not  illustrated  here, 
can  serve  multiple  objectives. 

In  the  example  illustrated  in  Figure  D.3,  Objectives  A  and  B  support  the  following 
goal.  Develop  a  shared  understanding  of  the  DES  and  its  application  within  and 
across  the  military  departments. 

As  noted  earlier,  by  using  this  process  iteratively,  we  identified  an  initial  set  of  objec¬ 
tives  and  goals.  Then,  from  the  perspective  of  achieving  the  goals,  we  examined  the 
set  of  objectives  and  the  relationships  we  had  posited  to  the  goals  to  assess  their 
completeness  and  specificity.  Similarly,  we  examined  the  desired  results  supporting 
the  objectives  and,  lastly,  we  examined  the  actions  that  supported  achieving  the  de¬ 
sired  results  to  assess  their  completeness  and  specificity. 
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Figure  D.3— Multiple  Desired  Results  and  Objectives  Supporting  a  Single  Goal 
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When  we  felt  comfortable  that  the  relationships  between  the  issues  and  the  initial 
sets  of  desired  results,  actions,  objectives,  and  goals  were  logical  and  complete,  we 
arrived  at  a  final  set  of  objectives.  Appendix  C  lists  all  of  the  issues  we  identified,  and 
Appendix  E  links  the  final  set  of  recommended  actions  (organized  into  ten  categories 
of  interventions)  with  the  eight  objectives  they  are  designed  to  support.  The  actions 
provide  specificity  for  achieving  the  stated  objectives.  The  final  set  of  objectives  and 
goals  is  shown  in  Figure  D.4. 
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Figure  D.4 — Multiple  Objectives  Contributing  to  Specific  Goals 


Appendix  E 

RELATING  ACTIONS  TO  OBJECTIVES 


This  appendix  highlights  the  link  between  the  interventions  for  more  consistent  ap¬ 
plication  of  disability  policy  (and  the  actions  that  constitute  them)  and  the  objectives 
those  actions  are  designed  to  support.  The  tables  in  the  first  half  of  this  appendix 
group  the  actions  and  the  DES  objective  each  action  supports  by  intervention  cate¬ 
gory.  The  tables  in  the  second  half  of  this  appendix  present  the  same  actions 
grouped  differently  for  easy  reference— and  the  intervention  category  of  each  by  DES 
objective. 

ASSISTANT  SECRETARY  OF  DEFENSE  DECISIONS  INTERVENTION 

As  portrayed  in  Table  E.l,  the  actions  composing  the  Assistant  Secretary  of  Defense 
Decisions  intervention  primarily  support  the  objectives  of  communicating  the  pur¬ 
pose  and  role  of  the  Disability  Evaluation  System  and  clarifying  and  promulgating 
policy  application  standards.  This  intervention  emphasizes  the  importance  of  the 
OSD  leadership  setting  a  clearly  articulated  direction  for  the  DES  to  ensure  consis¬ 
tent  application  of  disability  policy. 

POLICY  GUIDANCE  INTERVENTION 

Promulgating  policy  guidance  reifies  the  agreements  reached  as  part  of  the  Assistant 
Secretary  of  Defense  Decision  intervention.  This  guidance,  together  with  formal  is¬ 
suance  of  a  clearly  articulated  direction  for  conducting  business  and  definition  of  key 
concepts,  supports  the  objectives  of  communicating  the  purpose  and  role  of  the  DES 
and  clarifying  and  promulgating  policy  application  standards.  In  addition,  expansion 
of  certification  requirements  support  the  objectives  of  delivering  timely  services 
within  the  DES,  delivering  effective  services  to  customers,  and  monitoring  status  and 
performance.  Table  E.2  presents  the  policy  guidance  intervention  actions  and  their 
related  objectives. 

ORGANIZATIONAL  CHANGE  INTERVENTION 

Although  only  two  actions  fall  into  the  category  of  organizational  interventions,  as 
shown  in  Table  E.3,  they  are  critical  to  the  objectives  of  expediting  decisionmaking 
and  assigning  accountability. 
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Table  E.l 

Actions  and  Objectives  for  the  Assistant  Secretary  of  Defense  Decision  Intervention 

_ _ _ Action _ 

ASD/FMP-<iirect  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  a  DES  purpose  and  role 

ASD/FMP— direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  the  purpose  and  role  of  the  Medical 
Evaluation  Board. 

ASD/FMP— direct  the  Director.  Officer  and  Enlisted  Personnel  Clarify  and  promulgate  standards 

Management,  to  consult  with  a  small  group  of  experienced  DES 

experts  representing  the  military  departments’  PEBs  and  Office 

of  the  Surgeons  General  to  produce  recommendations  upon 

which  the  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  the  appropriate  time  frame  for 

initiating  Medical  Evaluation  Boards 

ASD/FMP  direct  the  Director,  Officer  and  Enlisted  Personnel  Clarify  and  promulgate  standards 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  standards  for  referring  Medical  Boards 
to  the  PEB  that  allow  for  variations  among  military  depart¬ 
ments  based  on  different  missions  and  requirements 

ASD/FMP-<Iirect  the  Director,  Officer  and  Enlisted  Personnel  Clarify  and  promulgate  standards 

Management,  to  consult  with  a  small  group  of  experienced  DES 

experts  representing  the  military  departments’  PEBs  and  Office 

of  the  Surgeons  General  to  produce  recommendations  upon 

which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  mechanisms  for  seamless  transmission 

of  medical  board  information  from  one  military  department  to 

another 

ASD/FMP— direct  the  Director,  Officer  and  Enlisted  Personnel  Clarify  and  promulgate  standards 

Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  fitness  standards  and  acceptable  varia¬ 
tion  among  the  services  based  on  different  missions  and 
requirements. 

ASD/FMP  direct  the  Director,  Officer  and  Enlisted  Personnel  Clarify  and  promulgate  standards 

Management,  to  consult  with  a  small  group  of  experienced  DES 

experts  representing  the  military  departments’  PEBs  and  Office 

of  the  Surgeons  General  to  produce  recommendations  upon 

which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  reasons  for  nondeployability  and  use  of 

nondeployability  in  determinations  of  fitness. 


_ DES  Objective _ 

Communicate  purpose  and  role  of 
DES 


Communicate  purpose  and  role  of 
DES 
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Table  E.l — Continued 


Action 

DES  Objective 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP  in  coordination  with  the  ASD/HA  and  ASD/RA 
can  decide  on  what  information  to  use  to  determine  fitness  and 
disability  ratings 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  a  consistent  period  of  time  among  the 
services  to  allow  for  service  member  election  of  options  follow¬ 
ing  a  PEB  (or  higher  level  appellate  review  board)  decision  or 
explain  the  differences  that  would  allow  for  due  process 

Clarify  and  promulgate  standards 

Clarify  and  promulgate  standards 

Table  E.2 

Actions  and  Objectives  for  the  Policy  Guidance  Intervention 

Action 

DES  Objective 

rnmmimirate  Dumose  and  role  of 

the  DBS 


OSD — develop  and  promulgate  clearly  stated  purpose  and  role 
of  the  Medical  Evaluation  Board  within  the  overall  DBS 

OSD— develop  and  promulgate  policy  defining  appropriate 
time  frame  for  initiating  Medical  Evaluation  Board 

OSD — develop  and  promulgate  clearly  stated  standards  for 

referring  medical  boards  to  a  PEB 

Services — develop  and  promulgate  clearly  stated  fitness 

standards 

OSD— develop  and  promulgate  clear  policy  regarding  reasons 
for  nondeployability  and  use  of  nondeployability  in  determina¬ 
tions  of  fitness 

OSD— develop  and  promulgate  clear  policy  regarding  what 
information  is  used  to  determine  fitness  and  disability  rating 

OSD — develop  and  promulgate  clearly  stated  policy  that 
provides  for  a  consistent  period  of  time  for  service  member 
election  of  options  across  the  services  or  a  rationale  for 
differences  in  policy  that  would  provide  due  process 

The  Department  of  the  Navy— assess  the  criteria  for  assigning 
service  members  to  Limited  Duty  in  the  context  of  determining 
the  appropriate  time  frame  for  Medical  Evaluation  Board 
referral 

OSD — develop  and  promulgate  clearly  stated  standards  for  the 
mechanisms  for  seamless  transmission  of  medical  board 
information  from  one  military  department  to  another 

OSD — develop  and  implement  certification  procedures 


Communicate  purpose  and  role  of 
the  DES 

Clarify  and  promulgate  standards 
Clarify  and  promulgate  standards 
Clarify  and  promulgate  standards 
Clarify  and  promulgate  standards 

Clarify  and  promulgate  standards 
Clarify  and  promulgate  standards 

Clarify  and  promulgate  standards 

Deliver  timely  services  within  the 
DES 

Deliver  timely  services  within  the 
DES 
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Table  E.2 — Continued 


Action 

DES  Objective 

OSD— test  primary  participants’  understanding  of  appropriate 
fields  of  knowledge;  require  certification 

Deliver  timely  services  within  the 

DES 

OSD — develop  certification  standards 

Deliver  effective  services  to 
customers 

OSD — develop  certification  standards  that  support  desired 
outcomes 

Monitor  status  and  performance 

Table  E.3 

Actions  and  Objectives  for  the  Organizational  Change  Intervention 

Action 

DES  Objective 

USD/PR— establish  procedures  and  a  forum  at  the  level  of  the 
Deputy  Assistant  Secretaries  of  Defense  and  their  counterparts 
in  the  military  departments 

Expedite  decisionmaking 

Secretary  of  each  military  department— designate  an  overall 
process  owner  for  that  department’s  DES 

Assign  accountability 

PERSONNEL  POLICY  INTERVENTION 

Personnel  policy  actions  support  the  objectives  of  clarifying  and  promulgating  stan¬ 
dards  and  assigning  accountability.  The  first  two  actions  shown  in  Table  E.4  require  a 
review  of  policies  outside  the  boundary  of  the  DES,  but  they  are  important, 
nonetheless,  because  of  the  incentives  they  may  create  within  the  system  to  "over- 
advocate”  for  the  patient.  The  third  and  fourth  actions  in  the  table  tie  individual 
performance  to  the  primary  participants’  impact  on  the  system. 

Table  E.4 


Actions  and  Objectives  for  the  Personnel  Policy  Intervention 


Action 

DES  Objective 

The  services — assess  the  difficulty  of  placing  service  members 
who  are  fit  but  not  deployable  into  units  that  can  utilize  their 
skills  and  experience 

Clarify  and  promulgate  standards 

The  OSD — articulate  an  explicit  policy  with  regard  to  fitness 
and  disability  ratings  for  a  service  member  who  is  nearing  20 
years  of  service 

Clarify  and  promulgate  standards 

OSD — review  the  impact  of  the  Defense  Authorization  Act  of 
1993  amendment  to  10  U.S.C.  1 142  that  requires  providing 
pre-separation  counseling  for  service  members  not  later  than 

90  days  before  separation,  as  it  applies  to  service  members 
undergoing  disability  evaluation 

Clarify  and  promulgate  standards 

Designated  process  owners— assess  the  performance  of  the 
military  department  PEB  approving  authority  and  the  MTF 
commanders  and  provide  the  assessment  to  the  officials  who 
write  their  performance  evaluation  reports 

Assign  accountability 
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PERSONNEL  MANAGEMENT  INTERVENTION 

Table  E.5  highlights  an  instance  in  which  a  single  action  supports  several  ohjectives. 
In  this  case,  the  assignment  of  PEB  approving  authorities  to  a  position  for  at  least  five 
years  supports  the  objectives  of  enhancing  communications,  delivering  timely  and 
effective  services,  and  expediting  decisionmaking.  In  addition,  assigning  PEBLOs  for 
longer  tours  and  monitoring  the  length  of  those  assignments  supports  the  objective 
of  delivering  effective  services  to  customers. 

Table  E.5 

Actions  and  Objectives  for  the  Personnel  Management  Intervention 


Action 

DES  Objective 

Military  departments— assign  PEB  approving  authority  for  a 
minimum  of  five  years 

Military  departments — ^increase  capabilities  of  PEBLOs  to 
generate  accurate  expectations  through  combination  of  experi¬ 
ence,  training,  and  information  support 

Military  departments — monitor  assignments 

Enhance  communications 

Deliver  timely  services  within  the 

DES 

Deliver  effective  services  to 
customers 

Expedite  decisionmaking 

Deliver  effective  services  to 
customers 

Deliver  effective  services  to 

customers 

TRAINING  INTERVENTION 

Although  Chapter  5  describes  the  major  recommendations  regarding  training,  two 
related  actions  support  the  objective  of  enhancing  communications  and  a  third  ac¬ 
tion  supports  two  other  objectives,  as  shown  in  Table  E  6. 

Table  E.6 

Actions  and  Objectives  for  the  Training  Intervention 

Action 

DES  Objective 

OASD/FMP— develop  and  deliver  training  designed  to  expedite 
medical  board  processing 

Disability  Advisory  Council — sponsor  cross -military  depart¬ 
ment  symposia  or  workshops 

Military  departments — conduct  annual  symposia  for  all  pri¬ 
mary  participant  populations  to  present,  review,  and  analyze 
military  department  DES  data  and  propose  corrective  actions 
and  identify  best  practices 

Deliver  timely  services  within  the 

DES 

Clarify  and  promulgate  standards 

Enhance  communications 

Enhance  communications 

INFORMATION  SOURCE  DEVELOPMENT  INTERVENTION 

Table  E.7  shows  the  importance  of  the  actions  constituting  the  information  source 
intervention— they  influence  the  attainment  of  five  objectives.  In  most  cases,  these 
actions  support  or  flow  from  actions  found  in  other  intervention  categories. 
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Table  E.7 

Actions  and  Objectives  for  the  Information  Source  Development  Intervention 


_ Action _ 

OSD — develop  a  brochure  and/or  Web  site  for  individuals  sep¬ 
arated  or  retired  for  disability  that  describes  the  service  mem¬ 
ber  s  rights,  benefits,  and  entitlements  and  the  significance  and 
consequences  of  the  determinations  reached,  including  a 
comprehensive  comparison  of  the  VA  and  the  DoD  disability 
systems 

Military  departments — publish  instruction  or  regulation  that 
describes  format  and  content  of  medical  boards 

OSD — develop  database  of  “best  practices"  in  the  DES 

OSD — establish  a  list  server  for  Medical  Evaluation  Board 
approving  authorities;  PEB  members  and  approving 
authorities;  and  for  PEBLOs 

MTF  commanders — send  a  sample  of  a  good  nonmedical 
assessment  commander's  letter  with  the  letter  notifying  the 
unit  commander  that  a  service  member  is  being  referred  to  the 
DES 

OSD — include  in  Web  site  examples  of  commander's  letters; 
Web-based  template  with  instant  transmission  to  PEBLO 

The  Office  of  the  Surgeons  General — update  medical  policy 
documents  to  match  OSD  and  military  department  disability 
policy  documents 

OSD — develop  an  information  source  that  primary  participants 
cannot  take  with  them  when  they  rotate  to  new  assignments 
and  that  invites  interaction  via  the  Internet 

OSD — develop  user-friendly  guide  to  narrative  summary 
requirements  [tests  and  measures)  for  the  major  five 
specialties;  instruct  physicians  to  use  it  when  dictating 
narrative  summaries 

OSD — develop  user-friendly  guide  to  narrative  summary 
requirements  (tests  and  measures)  for  all  diseases  or  injuries 
and  instruct  physicians  to  use  it  when  dictating  narrative 
summaries 

OSD — develop  a  tool  to  measure  customer  satisfaction  (survey) 

OSD— develop  information  tools  to  enhance  PEBLO’s  capabili- 
ties:  list  server,  lists  of  telephone  numbers,  Web  sites 


_ DES  Objective _ 

Communicate  purpose  and  role  of 
Disability  Evaluation  System 


Clarify  and  promulgate  standards 

Enhance  communications 
Enhance  communications 


Deliver  timely  services  within  the 
DES 


Deliver  timely  services  within  the 
DES 

Deliver  timely  services  within  the 
DES 


Deliver  timely  services  within  the 
DES 


Deliver  timely  services  within  the 
DES 


Deliver  timely  services  within  the 
DES 


Deliver  effective  services  to 
customers 

Deliver  effective  services  to 
customers 


MANAGEMENT  INFORMATION  SYSTEM  DEPLOYMENT  INTERVENTION 

The  actions  related  to  the  management  information  system  all  support  the  objective 
of  monitoring  DES  status  and  performance.  The  actions  derived  from  the  issues  re¬ 
garding  a  management  information  system  (shown  in  Table  E.8)  represent  a  small 
portion  of  the  overall  recommendations  regarding  the  structure  of  the  system,  which 
emphasize  the  value  of  a  top-down  approach  to  measuring  system  performance.  (See 
Chapter  6  for  a  discussion  of  the  specifications  of  the  management  information 
system.) 
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Table  E.8 

Actions  and  Objectives  for  the  Management  Information  System  Deployment  Intervention 


Action 

Objective 

OSD — develop  certification  standards  that  support  desired 

Monitor  DES  status  and  performance 

outcomes 

OSD — develop  a  reporting  framework  or  format  for  use  by  the 
MTF  commanders  and  the  Surgeons  General 

Monitor  DES  status  and  performance 

OSD — develop,  and  military  departments  employ,  a  monitor¬ 
ing  system  to  track  cases  from  narrative  summary  dictation  to 
MTF  commander  signature 

Monitor  DES  status  and  performance 

OSD — employ  consistent  processing  measures;  military 
departments  monitor  and  report  processing  time  for  medical 

Monitor  DES  status  and  performance 

boards 

Military  departments — collect  data  on  returned  medical 
boards  by  reason,  by  physician,  PEBLO,  and  unit 

Monitor  DES  status  and  performance 

commander;  report  this  data  to  the  MTF  commander 

OSD — develop  an  institutional  mechanism  for  quality  control 

Monitor  DES  status  and  performance 

or  assurance 

DES  process  owners — ^report  results  to  Secretaries  of  the  mili¬ 
tary  departments 

Monitor  DES  status  and  performance 

OSD — develop  mechanism  for  providing  information  on  the 
number  of  physicians  trained  to  Secretaries  of  the  military 
departments 

Monitor  DES  status  and  performance 

DES  process  owners — ^present  certification  data  to  Secretaries 
of  the  military  departments 

Monitor  DES  status  and  performance 

OSD  and  the  military  departments — develop  organizational 
capability  to  use  data  to  improve  system  operation 

Monitor  DES  status  and  perfoimance 

PROCESS  INTERVENTION 

Our  analysis  of  the  issues  related  to  the  disability  evaluation  “process”  suggested 
several  actions,  shown  in  Table  E.9,  which  complement  actions  in  other  intervention 
categories  that  support  the  objectives  of  delivering  timely  services  within  the  DES, 
assigning  accountability,  and  monitoring  status  and  performance. 

Table  E.9 

Actions  and  Objectives  for  the  Process  Intervention 

Action 

DES  Objective 

OSD — direct  the  military  departments  to  implement  a  proce-  Deliver  timely  services  within  the 

dure  whereby  a  Medical  Evaluation  Board  decision  to  forward  a  DES 

case  to  the  PEB  would  trigger  a  letter  from  the  MTF 

commander  to  the  unit  commander  explaining  the  unit 

commander's  role  in  the  process 

MTF  commanders — designate  and  train  one  physician  at  each  Deliver  timely  services  within  the 

MTF  to  dictate  all  narrative  summaries;  explore  other  options  DES 
for  focusing  expertise  in  writing  narrative  summaries 
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Table  E.9 — Continued 


Action 

DES  Objective 

MTF  commanders — designate  a  case  owner  (for  example,  in 
the  Air  Force,  the  referring  physician  owns  the  case);  PEBLOs 

Assign  accountability 

could  also  serve  this  function 

OSD  and  military  departments — develop  organizational  capa¬ 
bility  to  use  data  to  improve  system  operation 

Monitor  status  and  performance 

INCENTIVES  INTERVENTION 

We  identified  only  one  specific  action  as  an  incentive  intervention,  shown  in  Table 
E.IO.  However,  as  noted  in  Chapter  4,  expanded  recognition  and  reward  programs 
would  substantially  support  the  objective  of  assigning  accountability. 

Table  E.IO 

Action  and  Objective  for  the  Incentives  Intervention 


Action 

DES  Objective 

PEBs — publicly  recognize  the  best-performing  MTFs  annually 
with  an  award  of  excellence 

Assign  accountability 

LINKING  ACTIONS  TO  DES  OBJECTIVES 

Tables  E.ll  through  E.18  present  the  same  actions  that  appear  in  the  tables  in  the 
previous  sections.  In  this  group  of  tables,  however,  the  actions  and  their  associated 
intervention  categories  are  grouped  by  DES  objective. 

Table  E.ll 

Actions  and  Interventions  for  the  Communicate  Purpose  and  Role  of  Disability  Evaluation 

System  Objective 


_ _  Action _ 

ASD/FMP— direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  a  DES  purpose  and  role 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  the  purpose  and  role  of  the  Medical 
Evaluation  Board 


_ Category  of  Intervention 

Assistant  Secretary  of  Defense 
Decision 


Assistant  Secretary  of  Defense 
Decision 


OSD— develop  and  promulgate  purpose  and  role  of  the  DES  Policy  Guidance 

OSD— develop  and  promulgate  clearly  stated  purpose  and  role  Policy  Guidance 
of  the  Medical  Evaluation  Board  within  the  overall  DES 
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Table  E.ll — Continued 


_  Action _ Category  of  Intervention 

OSD — develop  a  brochure  and/  or  Web  site  for  individuals  sep-  Information  Source 

arated  or  retired  for  disability  that  describes  the  service  mem¬ 
ber's  rights,  benefits,  and  entitlements  and  the  significance  and 
consequences  of  the  determinations  reached,  including  a 

comprehensive  comparison  of  VA  and  DoD  disability  systems _ 


Table  E.12 

Actions  and  Interventions  for  the  Clarify  and  Promulgate  Standards  Objective 


_ Action _ 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  the  appropriate  time  frame  for 
initiating  Medical  Evaluation  Board 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  the  standards  for  referring  medical 
boards  to  the  PEB  that  allow  for  variations  among  military 
departments  based  on  different  missions  and  requirements 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  fitness  standards  and  acceptable  varia¬ 
tion  among  the  services  based  on  different  missions  and 
requirements 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 
ASD/RA,  can  decide  on  reasons  for  nondeployability  and  use  of 
nondeployability  in  determinations  of  fitness 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  vwth  the  ASD/HA  and 
ASD/RA,  can  decide  on  what  information  to  use  to  determine 
fitness  and  disability  ratings 


Category  of  Intervention 
Assistant  Secretary  of  Defense 
Decision 


Assistant  Secretary  of  Defense 
Decision 


Assistant  Secretary  of  Defense 
Decision 


Assistant  Secretary  of  Defense 
Decision 


Assistant  Secretary  of  Defense 
Decision 
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Table  E.12 — Continued 


Action 

Category  of  Intervention 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments’  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  a  consistent  period  of  time  among  the 
services  to  allow  for  service  member  election  of  options  follow¬ 
ing  a  PEB  (or  higher  level  appellate  review  board)  decision  or 
explain  the  differences  that  would  allow  for  due  process 

Assistant  Secretary  of  Defense 
Decision 

OSD — develop  and  promulgate  policy  defining  appropriate 
time  frame  for  initiating  Medical  Evaluation  Board 

Policy  Guidance 

OSD — develop  and  promulgate  clearly  stated  standards  for 
referring  medical  boards  to  a  PEB 

Policy  Guidance 

Services— develop  and  promulgate  clearly  stated  fitness 
standards 

Policy  Guidance 

OSD — develop  and  promulgate  clear  policy  regarding  reasons 
for  nondeployability  and  use  of  nondeployability  in  determina¬ 
tions  of  fitness 

Policy  Guidance 

OSD — develop  and  promulgate  clear  policy  regarding  what 
information  is  used  to  determine  fitness  and  disability  rating 

Policy  Guidance 

OSD — develop  and  promulgate  clearly  stated  policy  that 
provides  for  a  consistent  period  of  time  for  service  member 
election  of  options  across  the  services  or  a  rationale  for 
differences  in  policy  that  would  provide  due  process 

Policy  Guidance 

The  Department  of  the  Navy — assess  the  criteria  for  assigning 
service  members  to  Limited  Duty  in  the  context  of  determining 
the  appropriate  time  frame  for  Medical  Evaluation  Board 
referral 

Policy  Guidance 

Services — assess  the  difficulty  of  placing  service  members  who 
are  fit  but  not  deployable  into  units  that  can  utilize  their  skills 
and  experience 

Personnel  Policy 

OSD — articulate  an  explicit  policy  with  regard  to  fitness  and 
disability  ratings  for  service  member  who  is  nearing  20  years  of 
service 

Personnel  Policy 

OSD — review  the  impact  of  the  Defense  Authorization  Act  of 
1993  amendment  to  10  U.S.C.  1 142  that  requires  providing 
pre-separation  counseling  for  service  members  no  later  than 

90  days  before  separation,  as  it  applies  to  service  members 
undergoing  disability  evaluation 

Personnel  Policy 

Military  departments— publish  instruction  or  regulation  that 
describes  format  and  content  of  medical  boards 

Information  Source 

Table  E.  13 

Actions  and  Interventions  for  the  Enhance  Communications  Objective 

Action 

Category  of  Intervention 

Military  departments — assign  PEB  approving  authorities  for  a 
minimum  of  five  years 

Personnel  Management 

OASD/FMP — develop  and  deliver  training  designed  to  expedite 
medical  board  processing 

Training 
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Table  E.13 — Continued 


Action 

Category  of  Intervention 

Disability  Advisory  Council — sponsor  cross-military  depart¬ 
ment  symposia  or  workshops 

Training 

Military  departments — conduct  annual  symposia  for  all  of  the 
primary  participant  populations  to  present,  review,  and  ana¬ 
lyze  military  department  DES  data  and  propose  corrective 
actions  and  identify  best  practices 

Training 

OSD — develop  database  of  “best  practices”  in  the  DES 

Information  Source 

OSD — establish  a  list  server  for  Medical  Evaluation  Board 
approving  authorities;  PEB  members  and  approving 
authorities;  and  for  PEBLOs 

Information  Source 

Table  E.14 

Actions  and  Interventions  for  the  Deliver  Timely  Services  Within  the  Disability  Evaluation 

System  Objective 

Action 

Category  of  Intervention 

ASD/FMP — direct  the  Director,  Officer  and  Enlisted  Personnel 
Management,  to  consult  with  a  small  group  of  experienced  DES 
experts  representing  the  military  departments'  PEBs  and  Office 
of  the  Surgeons  General  to  produce  recommendations  upon 
which  ASD/FMP,  in  coordination  with  the  ASD/HA  and 

ASD/RA,  can  decide  on  mechanisms  for  seamless  transmission 
of  medical  board  information  from  one  military  department  to 
another 

Assistant  Secretary  of  Defense 
Decision 

OSD — develop  and  promulgate  clearly  stated  standards  for  the 
mechanisms  for  seamless  transmission  of  medical  board 
information  from  one  military  department  to  another 

Policy  Guidance 

OSD — develop  and  implement  certification  procedures 

Policy  Guidance 

OSD — ^test  primary  participants’  understanding  of  appropriate 
fields  of  knowledge;  require  certification 

Policy  Guidance 

Military  departments — assign  PEB  approving  authorities  for  a 
minimum  of  five  years 

Personnel  Management 

OASD/FMP — develop  and  deliver  training  designed  to  expedite 
medical  board  processing 

Training 

Office  of  the  Surgeons  General — ^update  medical  policy 
documents  to  match  OSD  and  military  department  disability 
policy  documents 

Information  Source 

OSD — develop  an  information  source  that  primary  participants 
can  not  take  with  them  when  they  rotate  to  new  assignments 
and  that  invites  interaction  via  the  Internet 

Information  Source 

MTF  commanders — send  a  sample  of  a  good  nonmilitary 
assessment  commander’s  letter  with  the  letter  notifying  the 
unit  commander  that  a  service  member  is  being  referred  to  the 
DES 

Information  Source 

OSD — ^include  in  Web  site  examples  of  commander’s  letter; 
Web-based  template  with  instant  transmission  to  PEBLO 

Information  Source 
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Table  E.14 — Continued 

_ Action _ 

OSD — develop  user-friendly  guide  to  narrative  summary 
requirements  (tests  and  measures)  for  the  major  five 
specialties;  instruct  physicians  to  use  it  when  dictating 
narrative  summaries 

OSD— develop  user-friendly  guide  to  narrative  summary  Information  Source 

requirements  (tests  and  measures)  for  all  diseases  or  injuries 
and  instruct  physicians  to  use  it  when  dictating  narrative 
summaries 

MTF  commanders—designate  and  train  one  physician  at  each  Process 
MTF  to  dictate  all  narrative  summaries;  explore  other  options 
for  focusing  expertise  in  writing  narrative  summaries 

OSD— direct  the  military  departments  to  implement  a  proce-  Process 

dure  whereby  a  Medical  Evaluation  Board  decision  to  forward  a 

case  to  the  PEB  would  trigger  a  letter  from  the  MTF 

commander  to  the  unit  commander  stating  the  intent  to 

process  the  member  through  the  DES,  detailing  the  process, 

and  explaining  the  unit  commander's  role  in  the  process 

Table  E,15 

Actions  and  Interventions  for  the  Deliver  Effective  Services  to  Customers  Objective 

_ Action _ 

OSD — develop  certification  procedures 

Military  departments — increase  capabilities  of  PEBLOs  to 
generate  accurate  expectations  through  combination  of  experi 
ence,  training,  and  information  support 

Military  departments— assign  PEB  approving  authorities  for  a 
minimum  of  five  years 

Military  departments — monitor  assignments 

OSD— develop  information  tools  to  enhance  PEBLO’s  perfor-  Information  Source 
mance  capabilities:  list  server,  lists  of  telephone  numbers,  Web 
site 

OSD— develop  a  tool  to  measure  customer  satisfaction  (survey)  Information  Source 


Category  of  Intervention 
Policy  Guidance 

Personnel  Management 

Personnel  Management 
Personnel  Management 


Table  E.16 

Actions  and  Interventions  for  the  Expedite  Decisionmaking  Objective 

_ _ 

USD/PR — establish  procedures  and  a  forum  at  the  level  of  the 
Deputy  Assistant  Secretaries  of  Defense  and  their  counterparts 
in  the  military  departments 


_ Category  of  Intervention 

Organizational  Change 
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Table  E.17 

Actions  and  Interventions  for  the  Assign  Accountability  Objective 


Action 

Category  of  Intervention 

Secretary  of  each  military  department — designate  an  overall 
process  owner  for  that  department’s  DES 

Organizational  Change 

Designated  process  owners — assess  the  performance  of  the 
military  department  PEB  approving  authority  and  MTF 
commanders  and  provide  the  assessment  to  the  officials  who 
write  their  performance  evaluation  reports 

Personnel  Policy 

MTF  commanders — designate  a  case  owner  (for  example,  in 
the  Air  Force  the  referring  physician  owns  the  case);  PEBLOs 

Process 

could  also  serve  this  function. 

PEBs— publicly  recognize  the  best-performing  MTFs  annually 
with  an  award  of  excellence 

Incentive 

Table  E.  18 

Actions  and  Interventions  for  the  Monitor  Disability  Evaluation  System  Status  and 

Performance  Objective 


_ Action _ 

OSD — develop  certification  standards  that  support  desired 
outcomes 

OSD — develop  a  reporting  framework  or  format  for  use  by  the 
MTF  commanders  and  the  Surgeons  General 

OSD — develop  and  employ  a  monitoring  system  to  track  cases 
from  narrative  summary  dictation  to  MTF  commander 
signature 

OSD — employ  consistent  processing  measures;  military 
departments,  monitor  and  report  processing  time  for  medical 
boards 

Military  departments — collect  data  on  returned  medical 
boards  by  reason,  physician,  PEBLO,  and  unit  commander; 
report  this  data  to  the  MTF  commander 

OSD — develop  an  institutional  mechanism  for  quality  control 
or  assurance 

DES  process  owners — ^report  results  to  Secretaries  of  the 
military  departments 

OSD — develop  mechanism  for  providing  information  on  the 
number  of  physicians  trained  to  Secretaries  of  the  military 
departments 

DES  process  owners — ^present  certification  data  to  Secretaries 
of  the  military  departments 

OSD  and  the  military  departments — develop  organizational 
capability  to  use  data  to  improve  system  operation _ 


Category  of  Intervention 
Policy  Guidance 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Management  Information  System 
Deployment 

Process 


Appendix  F 

COMPARING  THE  ISSUES-DRIVEN  AND  PURPOSE-DRIVEN 

ANALYTIC  APPROACHES 


In  this  appendix,  we  compare  the  two  fundamental  approaches  that  we  used  to  study 
the  Disability  Evaluation  System— the  purpose-driven,  top-down  approach,  as 
described  in  Chapters  3,  5,  and  6,  and  the  issues-driven,  bottom-up  approach,  as  de¬ 
scribed  in  Chapter  4.  We  regard  any  system  as  a  set  of  interrelated  actions  connected 
in  a  specific  order,  which  presents  a  logical  plan  for  linking  various  actions  in  order 
to  accomplish  certain  desired  outcomes. 

Both  of  the  approaches  we  used  in  this  study  impart  order  to  the  numerous  actions 
that  collectively  make  up  the  DES.  Each  approach  relies  on  different  but  related  con¬ 
structs  to  present  the  proposed  actions,  which  are  bundled  within  categories  of  in¬ 
terventions,  and  an  overall  plan  for  achieving  the  desired  outcomes.  However,  issues 
form  the  empirical  basis  for  action  in  the  bottom-up  approach,  whereas  actual  out¬ 
comes  measured  against  desired  outcomes  and  the  stated  system  purpose  form  the 
basis  for  action  in  the  top-down  approach.  This  appendix  describes  the  relationships 
among  the  various  constructs  within  the  context  of  these  two  approaches. 

PURPOSE-DRIVEN  APPROACH 

Ideally,  we  would  have  preferred  to  employ  a  single  top-down  approach,  such  as  il¬ 
lustrated  in  Figure  F.l.  Such  an  approach,  however,  would  require  a  commonly 
agreed  upon  purpose  for  the  DES,  a  set  of  desired  system  outcomes,  and  an  informa¬ 
tion  system  to  measure  actual  outcomes  against  desired  outcomes.  In  that  context, 
the  observed  differences  between  desired  and  actual  outcomes  would  lead  to  the 
identification  and  recommendation  of  interventions  to  eliminate  those  differences. 
As  a  key  intervention,  effective  training,  in  particular,  must  be  based  on  both  specific 
and  measurable  training  objectives  tied  to  the  desired  system  outcomes  and  on  an 
assessment  of  how  well  the  objectives  are  currently  being  achieved. 

In  the  top-down  approach,  when  the  desired  outcomes  are  achieved,  the  system  ac¬ 
complishes  its  purpose.  The  desired  outcomes  suggest  the  kind  of  data  the  manage¬ 
ment  information  system  must  gather  and  also  the  competencies  the  primary  partic¬ 
ipants  need  to  perform  their  jobs  effectively.  By  comparing  actual  outcomes  with 
desired  outcomes,  the  management  information  system  establishes  the  basis  for 
training  tmd  other  interventions. 
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Figure  F.l — The  Purpose-Driven,  Top-Down  Approach 


Chapter  3  of  this  report  describes  a  suggested  purpose  and  five  supporting  outcomes. 

We  use  the  outcomes  in  two  ways: 

•  Four  of  the  five  outcomes  lead  directly  (and  the  fifth  leads  indirectly)  to  the 
identification  of  competencies  needed  by  the  primary  participants  in  order  for 
them  to  accomplish  the  purpose  of  the  DES.  We  base  the  design  of  training 
intervention  (discussed  in  Chapter  5)  on  the  bodies  of  knowledge  underlying 
these  competencies. 

•  All  five  outcomes  lead  to  the  functional  specifications  for  an  information  system, 
as  discussed  in  Chapter  6.  This  system,  once  deployed,  will  provide  the  means  to 
identify  problems  in  the  consistent  application  of  disability  policy  over  time  and 
provide  a  more  concrete  foundation  upon  which  to  shape  training  in  the  future. 


ISSUES-DRIVEN  APPROACH 

Based  on  our  conversations  with  primary  participants  and  information  gathered  at 
workshops  and  training  sessions,  we  identified  dozens  of  specific  problem  areas,  or 
issues,  related  to  the  consistent  application  of  disability  policy  that  exist  within  the 
current  DES.  To  develop  recommendations  for  immediate  execution,  we  employed  a 
goal  fabric  framework — an  issues-driven,  bottom-up  approach  as  illustrated  in 
Figure  F.2— that  capitalized  on  the  data  we  gathered. 
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This  approach  helped  identify  the  desired  results— what  we  would  observe  (related 
to  the  issue)  if  the  difference  were  eliminated  or  the  problem  solved— and  specific 
actions  that  would  bring  about  the  desired  results.  To  ensure  that  the  actions  are 
comprehensive,  this  approach  aligns  desired  results  in  terms  of  the  objectives  they 
satisfy  and  aligns  objectives  in  terms  of  the  goals  they  satisfy.  The  final  product  of  this 
analysis,  ten  categories  of  interventions  (composed  of  similar  actions),  represents  a 
comprehensive  plan  for  moving  toward  consistent  application  of  disability  policy. 


INTEGRATING  THE  APPROACHES 

The  issues-driven  and  purpose-driven  approaches  are  actually  not  as  different  as 
they  might  first  appear  to  be.  Figure  F.3  suggests  their  relative  similarities. 

The  goals  and  objectives  of  the  bottom-up  approach  function  in  much  the  same 
manner  as  the  DBS  purpose  does  in  the  top-down  approach,  and  the  desired  results 
in  one  are  similar  to  the  desired  outcomes  in  the  other.  As  stated  earlier  in  this  ap¬ 
pendix,  issues  form  the  empirical  basis  for  action  in  the  bottom-up  approach, 
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Near-Term  Plan 
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Figure  F.2 — The  Issues-Driven,  Bottom-Up  Approach 
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Near-Term  Plan 


BAHDMR1228'F.3 


Figure  F.3 — Comparing  the  Two  Approaches 


whereas  actual  outcomes  as  measured  by  the  management  information  system  serve 
that  function  in  the  top-down  approach.  Actions  grouped  into  categories  of  inter¬ 
ventions  correspond  to  the  training,  management  information  system,  and  other  in¬ 
terventions  in  the  top-down  approach. 

We  employed  each  approach  to  accomplish  different  tasks.  The  bottom-up  approach 
was  very  fruitful  in  identifying  shortfalls  in  the  consistent  application  of  disability 
policy,  which  the  OSD  can  rectify  immediately  by  implementing  specific  interven¬ 
tions.  This  approach  also  affirmed  the  necessity  of  pursuing  standardized  training  for 
the  primary  participants  and  developing  a  management  information  system  to 
monitor  the  performance  of  the  system. 

Although  the  bottom-up  approach  is  the  basis  for  the  recommended  interventions 
presented  in  Chapter  4,  it  does  not  provide  enough  of  a  foundation  for  actually  de¬ 
signing  the  training  intervention  or  developing  the  functional  specifications  for  the 
management  information  system  intervention.  For  these  latter  tasks,  we  used  the 
top-down  approach. 


Appendix  G 

DISABILITY  EVALUATION  SYSTEM  TRAINING  TOPICS  FOR 
PRIMARY  PARTICIPANT  POPULATIONS  ACROSS 

MILITARY  DEPARTMENTS 


As  noted  in  Chapter  5,  the  suggested  purpose  of  the  DES  and  set  of  desired  outcomes 
shaped  the  suggested  performance  competencies  for  primary  participant  popula¬ 
tions,  which  in  turn  shaped  the  selection  of  DES  topics  for  training  content.  Chapter 
5  groups  the  DES  topics  into  six  distinct  training  packages  with  each  containing  a 
specific  body  of  DES  knowledge  for  the  relevant  primary  participant  population  clus¬ 
ters.  In  contrast,  Table  G.l  presents  the  raw  data  that  served  as  the  basis  for  the 
training  packages  presented  in  Chapter  5. 

Table  G.l  contains  a  comprehensive  list  of  DES  topics  and  our  assessment  of  which 
primary  participant  populations  require  knowledge  of  a  given  topic.  The  topics  are 
listed  in  the  order  in  which  we  collected  the  respective  data.  This  data  array  informed 
the  organization  and  development  of  the  content  of  the  six  training  packages. 

We  gleaned  much  of  what  appears  in  Table  G.l  directly  from  OSD  disability  policy 
documents,  augmented  by  existing  military  department  syllabi  and  policy  docu¬ 
ments,  and  through  observation  of  military  department  training  programs  and 
interviews  with  primary  participants.  A  few  of  the  topics  listed  in  the  table — such  as 
“Difference  Between  VA  and  DoD  DES”— came  directly  from  our  analysis  and  other 
recommended  interventions  noted  in  this  report. 

Note:  The  full  range  of  information  included  within  some  knowledge  areas  is  shown 
in  the  table  in  italics.  Not  all  primary  participant  populations  require  knowledge  in 
every  DES  topic  area. 
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Table  G.l 

Initial  Formulation  of  Disability  Evaluation  System  Topics  for  Primary  Participant  Populations 
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Appendix  H 

DESCRIPTION  OF  METRICS  FOR  ASSESSING 
DISABILITY  EVALUATION  SYSTEM  PERFORMANCE 


This  appendix  describes  the  full  set  of  performance  metrics  proposed  in  Chapter  6. 
The  metrics  described  in  the  first  two  sections  of  this  appendix  focus 
on  the  outcome  measures  directly  related  to  the  assessment  of  how  well  the  DBS 
fulfills  service  member  and  military  service  expectations;  the  metrics  described  in  the 
last  two  sections  focus  on  the  output  measures  related  to  the  two  interventions 
discussed  in  Chapter  6— process  improvements  and  enhanced  primary  participant 
competency. 

METRICS  DERIVED  FROM  SERVICE  MEMBER  EXPECTATIONS 

As  described  in  Chapter  6,  two  outcome  measures — case  variability  and  number  of 
appeals — derive  from  how  well  the  DBS  fulfills  service  member  expectations. 

Metrics  for  the  Case  Variability  Outcome  Measure 

We  recommend  the  following  metrics  to  assess  the  case  variability  outcome  measure: 
(1)  distribution  of  medical  boards  by  diagnostic  category;  (2)  statistical  analysis  of 
dispositions  (fitness,  rating,  and  personnel  action)  for  major  diagnostic  categories; 
and  (3)  statistical  analysis  of  dispositions  (fitness,  rating,  and  personnel  action)  for 
special  diagnostic  categories  (for  example,  HIV). 

These  metrics  address  the  relatively  aggregate  question  of  whether  the  military  de¬ 
partments  are  applying  disability  policy  consistently.  The  distribution  of  medical 
boards  by  diagnostic  category  is  useful  primarily  for  setting  the  stage  or  the  context 
for  the  statistical  analyses  of  dispositions;  it  also  helps  to  identify  the  effect  of  differ¬ 
ent  conditions  of  service  among  the  military  services.  The  statistical  analyses  are  the 
primary  means  of  assessing  consistent  application  of  disability  policy. 

The  OSD  should  obtain  quarterly  data  for  the  metric,  distribution  of  medical  boards 
by  diagnostic  category,  using  the  medical  boards  sent  to  the  Informal  PBB  in  the  pre¬ 
vious  quarter  as  a  basis  for  the  metric.  Bach  military  department’s  trend  serves  as  the 
primary  benchmark  because  different  conditions  of  service  in  each  of  the  depart¬ 
ments  make  comparisons  among  the  departments  less  meaningful.  The  data  for  this 
metric  should  be  collected  using  an  automated  system  or  a  hard  copy  form  included 
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in  the  medical  board  as  the  Informal  and  Formal  PEBs  render  a  decision  on  each 
medical  board. 

The  statistical  analyses  differ  from  the  other  metrics  we  recommend.  These  metrics 
need  to  be  developed  because  they  are  not  part  of  the  system.  We  believe  an  inde¬ 
pendent  organization,  such  as  the  DoD  Office  of  the  Inspector  General,  should  per¬ 
form  the  analysis.!  We  recommend  drawing  a  random  sample  from  medical  boards 
on  which  the  Informal  and  Formal  PEBs  rendered  decisions  in  the  previous  year.  To 
ensure  adequate  sample  sizes,  these  analyses  should  be  performed  only  for  medical 
boards  from  the  top  five  diagnostic  categories. 

The  analysis  should  test  the  hypothesis  that  a  difference  exists  in  the  dispositions 
regarding  fitness,  rating,  or  personnel  action  for  service  members  with  the  same  dis¬ 
abling  conditions.  This  hypothesis  should  be  tested  within  and  across  military  de¬ 
partments.  The  appropriate  benchmarks  are  military  department  trends  in  compari¬ 
son  with  the  overall  DoD  standard  of  no  significant  difference  within  or  among 
military  departments. 

Special  diagnostic  categories  (for  example,  service  members  who  are  diagnosed  as 
HIV-positive)  may  require  a  similar  analysis  of  dispositions  from  time  to  time.  These 
analyses  should  be  conducted  as  needed. 

Metrics  for  the  Number  of  Appeals  Outcome  Measure 

We  recommend  the  following  metrics  for  assessing  the  number  of  appeals  outcome 
measure:  (1)  percent  of  Informal  PEB  decisions  appealed  for  fitness;  (2)  percent  of 
Informal  PEB  decisions  appealed  for  rating;  (3)  percent  of  Formal  PEB  decisions  ap¬ 
pealed  for  fitness;  (4)  percent  of  Formal  PEB  decisions  appealed  for  rating;2  and  (5) 
the  percent  of  appeals  overturned  for  each  of  these  categories. 

The  underlying  premise  is  that  the  number  of  appeals  serves  as  a  proxy  for  the  level 
of  satisfaction  with  the  process,  within  a  particular  part  of  the  DES  or  for  a  particular 
military  department.  Increasing  appeals  could  suggest  growing  service  member  dis¬ 
satisfaction  with  the  operation  of  the  DES;  decreasing  appeals  could  suggest  that  a 
military  department  has  implemented  a  process  improvement  from  which  other 
military  departments  or  other  parts  of  a  military  department  DES  could  benefit. 

The  OSD  should  gather  quarterly  summary  data  from  the  military  departments  using 
as  a  basis  medical  boards  that  reflect  Informal  and  Formal  PEB  decisions  rendered  in 
the  previous  quarter.  The  appropriate  benchmarks  are  military  department  trends 
and  comparisons  with  the  overall  DoD  average.  The  data  for  these  metrics  should  be 
collected  using  an  automated  system  or  a  hard  copy  form  included  in  the  medical 


Ja  similar  analysis  was  conducted  to  support  a  DoD  Inspector  General  Audit  Report  on  "Medical 
Disability  Discharge  Procedures"  in  June  1992. 


^For  example,  for  Forrnal  PEB  decisions  on  100  medical  boards,  service  members  appealed  four  for 
disagreement  with  the  fitness  determination,  27  for  disagreement  with  the  rating  decision,  and  12  for 
disagreement  with  both;  or,  4, 27,  and  12  percent,  respectively. 
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board  as  the  Informal  PEB  and  Formal  PEB  renders  a  decision  on  each  medical 
board. 

In  addition,  because  service  member  perceptions  may  be  as  important  as  empirical 
data  in  assessing  whether  service  members  believe  their  expectations  are  being  ful¬ 
filled,  we  recommend  several  other  general  metrics  designed  to  investigate  this  out¬ 
come  measure  more  directly:  percent  of  service  members  satisfied  with  the  disposi¬ 
tion  decision;  percent  of  service  members  satisfied  with  the  process  (timeliness, 
courtesy,  responsiveness,  and  assistance);  percent  of  service  members  satisfied  that 
they  received  due  process;  number  of  congressionals  (letters  written  by  service 
members  to  their  representatives  in  Congress);  GAO  reports;  and  IG  reports. 

We  recommend  that  the  OSD  develop  a  DoD-wide  survey  that  the  military  depart¬ 
ments  can  administer  to  all  service  members  who  complete  processing  through  the 
DES.  The  purpose  of  this  survey  is  to  capture  the  service  members’  perceptions  re¬ 
garding  the  first  three  of  these  metrics:  percent  of  service  members  that  are  satisfied 
with  the  disposition  decision;  percent  of  service  members  that  are  satisfied  with  the 
process  (timeliness,  courtesy,  responsiveness,  and  assistance);  and  percent  of  service 
members  that  are  satisfied  that  they  received  due  process.  Many  of  the  metrics 
described  later  in  this  appendix  provide  insight  into  what  actually  happens  with 
components  of  the  DES  that  affect  service  members.  The  survey  will  provide  a 
(lagged)  link  between  interventions  in  the  DES  and  their  perceived  impact  on  service 
members.  We  recommend  a  100-percent  survey,  with  the  results  analyzed  annually. 
The  results  should  be  benchmarked  against  trends  and  explicit  DoD  standards. 

Letters  from  senators  and  representatives  sent  to  the  DoD  on  behalf  of  service  mem¬ 
bers  generally  indicate  a  significant  level  of  dissatisfaction  with  the  system.  Any  ser¬ 
vice  member  who  has  exhausted  administrative  avenues  of  relief  for  a  perceived  in¬ 
justice  and  chooses  to  take  his  or  her  case  to  a  member  of  the  Congress  has  expressed 
a  level  of  dissatisfaction  that  deserves  special  attention.  Data  for  this  metric  is  easy  to 
collect  directly  within  the  correspondence  management  system.  We  recommend 
annual  assessments,  broken  out  by  military  department,  that  are  benchmarked 
against  trends  and  DoD  averages. 

GAO  and  IG  reports  represent  ad  hoc  metrics  that  can  provide  additional  insight  into 
the  DES,  and  IG  reports  can  be  commissioned  to  focus  on  particular  issues.  Like  the 
pleas  to  members  of  Congress,  these  reports  may  also  indicate  a  certain  level  of  dis¬ 
satisfaction  with  the  system,  although  we  believe  they  will  be  too  small  in  number  to 
draw  any  conclusions  from  them. 

METRICS  DERIVED  FROM  MILITARY  SERVICE  EXPECTATIONS 

Two  general  outcome  measures  derive  from  how  well  the  DES  fulfills  service  expec¬ 
tations:  total  system  cost  and  time  to  replace  an  unfit  service  member. 
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Metrics  for  the  Total  System  Cost  Outcome  Measure 

We  recommend  the  following  metrics  for  assessing  the  total  system  cost  outcome 
measure:  (1)  total  resources  for  the  operation  of  the  DES;  (2)  pay  and  allowances  for 
service  members  not  performing  their  duties;  and  (3)  the  cost  of  disability  severance 
pay. 

The  underlying  issue  these  metrics  address  is  the  burden  the  DES  places  on  a  military 
department.  The  military  departments  strive  to  minimize  these  costs  to  be  consistent 
with  the  goal  of  accomplishing  the  purpose  of  the  Disability  Evaluation  System.  We 
recommend  reporting  cost  data  annually  and  benchmarking  the  data  against  trends. 

The  total  resources  devoted  to  operating  the  DES  is  an  important  indicator  of  how 
much  of  a  direct  burden  the  DES  places  on  a  military  department:  this  metric  is  im¬ 
portant  also  because  it  forms  the  basis  of  the  metrics  we  recommend  for  assessing 
productivity.  The  total  resources  metric  is  an  aggregation  of  pay  and  allowances  or 
salaries  of  military  and  civilian  primary  participants  in  the  DES;  information  system 
costs;  training  costs;  and  operations  and  maintenance  costs  (other  than  training  and 
civilian  salaries). 3  Pay  and  allowances  for  members  not  performing  their  duties 
(those  who  have  entered  the  DES  and  been  removed  from  their  unit)  indicate  the 
opportunity  cost  of  a  service  member  in  the  DES.  The  longer  the  service  member  re¬ 
mains  in  the  system,  the  higher  the  cost.  This  metric  places  a  value  on  processing 
time  and  allows  for  a  comparison  between  the  total  resources  devoted  to  operating 
the  system  and  the  cost  of  interventions  designed  to  shorten  the  processing  time. 


Metrics  for  the  Time  to  Replace  an  Unfit  Service  Member 
Outcome  Measure 

We  recommend  the  following  metrics  for  assessing  the  time  to  replace  an  unfit  service 
member  outcome  measure:  (1)  for  service  members  returned  to  duty,  average  total 
tirne  from  referral  to  an  MTF  to  return  to  duty;  (2)  for  service  members  separated  or 
retired,  average  total  time  from  referral  to  an  MTF  to  termination;  and  (3)  average 
total  time  on  the  TDRL,  broken  out  by  the  diagnostic  category.  We  recommend  up¬ 
dating  each  of  these  metrics  quarterly. 

These  time-to-replace  metrics  focus  on  the  key  contributor  to  service  satisfaction  (or 
dissatisfaction)— the  time  it  takes  to  replace  a  service  member  who  is  no  longer  able 
to  function  as  part  of  a  unit.  The  metrics  address  various  obstacles  that  stand  in  the 
way  of  a  commander  initiating  a  request  for  a  replacement.  Although  important  to 
the  individual  commander,  we  do  not  include  in  the  metrics  the  time  it  takes  to  ob¬ 
tain  a  replacement  through  the  personnel  system  because  that  system  is  not  part  of 
the  DES. 

The  first  two  time-to-replace  metrics,  the  average  time  from  referral  to  the  military 
ti  eatment  facility  until  return  to  duty  and  the  average  time  from  referral  to  the  mili- 


3 

These  components  of  total  resources  are  used  as  individual  metrics  when  assessing  performance 
measures  for  enhanced  primary  participant  competencies,  as  discussed  later  in  this  appendix. 
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tary  treatment  facility  until  termination,  are  computed  based  on  information  in  the 
medical  board  and  reported  using  an  automated  system  or  a  hard  copy  form 
attached  to  the  medical  board.  These  metrics  should  be  broken  out  by  diagnostic 
category  and  phase  of  the  DES  (Medical  Evaluation  Board,  Physical  Evaluation 
Board,  or  post-PEB  appellate  review).  Cases  in  which  the  service  member  is  returned 
to  duty  or  terminated  or  removed  from  the  TDRL  in  the  previous  quarter  form  the 
basis  of  the  monitored  population.  The  average  times  are  benchmarked  against 
historical  trends.  A  significant  increase  in  the  average  time  to  process  cases  should 
lead  to  an  investigation  of  its  underlying  causes.  To  accommodate  such  an  activity, 
the  data  should  be  collected  in  enough  detail  to  allow  for  an  inspection  of  the 
distribution  of  processing  times. 

We  recommend  similar  metrics  for  service  members  placed  on  limited  duty.  Several 
primary  participants  and  other  officials  we  interviewed  expressed  concern  with  the 
number  of  service  members  placed  on  limited  duty,  both  before  and  after  being 
referred  to  the  DES.  In  recognizing  that  the  limited  duty  determination  is  not  a  part 
of  the  DES,  we  recommend  a  metric  that  separately  monitors  the  distribution  of  time 
on  limited  duty  before  and  after  referral  to  the  DES,  with  an  eye  toward  ensuring  that 
the  referral  is  accomplished  at  the  appropriate  time. 

METRICS  RELATED  TO  ENHANCING  PRIMARY  PARTICIPANT 
COMPETENCY 

Three  general  output  measures  capture  the  effect  of  interventions  targeted  at  en¬ 
hancing  primary  participant  competencies:  productivity,  cost  per  medical  board  de¬ 
cision,  and  percent  of  primary  participants  certified.  The  input  measure,  total  re¬ 
sources,  augments  the  output  measures. 

Metrics  for  the  Productivity  Output  Measure 

We  recommend  the  following  metrics  for  assessing  the  productivity  output  measure: 
(1)  medical  board  decisions  rendered  per  Informal  PEB  member;  (2)  medical  board 
decisions  rendered  per  Formal  PEB  member;  (3)  medical  board  processing  com¬ 
pleted  per  full-time  PEBLO  and  PEB  administrative  action  officer  assigned  to  the 
PEB;  (4)  primary  participant  satisfaction,  by  primary  participant  population;  and  (5) 
turnover,  by  primary  participant  population. 

These  metrics  provide  insight  into  the  effectiveness  of  primary  participants  in  the 
system.  For  example,  declining  productivity  should  theoretically  lead  to  decreased 
service  member  and  service  satisfaction.  Monitoring  this  metric  and  acting  upon 
changes  enables  leaders  to  implement  corrective  action  in  time  to  head  off  decreased 
satisfaction.  In  other  words,  productivity  is  a  leading  indicator  of  service  member 
and  service  satisfaction. 

Metrics  for  the  other  two  output  measures  for  enhancing  primary  participant 
competency,  cost  per  medical  board  decision  and  percent  of  primary  participants 
certified,  and  the  input  measure  total  resources  devoted  to  the  Disability  Evaluation 
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System^  are  leading  performance  indicators  foretelling  change  in  the  lagging  indica¬ 
tor  productivity f  and  suggest  where  interventions  may  be  most  effective. 

For  the  first  three  productivity  metrics,  the  military  departments  should  pull  data 
from  Informal  and  Formal  PEB  decisions  rendered  and  total  medical  boards  pro¬ 
cessed  in  the  previous  quarter  together  with  current  manning  (staffing)  data  and  re¬ 
port  the  summary  results  quarterly.  The  results  should  be  represented  as  the  quo¬ 
tient  of  the  number  of  decisions  rendered  in  the  previous  quarter  and  the  average 
number  of  Informal  PEB  members,  Formal  PEB  members  or  full-time  PEBLOs,  and 
PEB  administrative  action  officers.  These  metrics  should  be  benchmarked  against 
trends  and  DoD  averages. 

The  fourth  and  fifth  productivity  metrics-— pr/moty  participant  satisfaction  by  pri¬ 
mary  participant  population  and  turnover  by  primary  participant  population— are 
interrelated,  with  turnover  being  a  manifestation  of  the  level  of  satisfaction  in  some 
cases.  We  recommend  an  annual  100-percent  survey  of  primary  participant 
satisfaction,  summarized  by  primary  participant  populations  (specifically.  Medical 
Evaluation  Board  members  and  approving  authorities,  PEBLOs,  PEB  administrative 
action  officers,  and  approving  authorities).  The  surveys  should  probe  for  the  source 
of  satisfaction  or  dissatisfaction  through  structured  multiple-choice  questions,  and 
solicit  suggestions  for  ways  to  improve  the  operation  of  the  system  through  open- 
ended  questions.  Summary  statistics  on  the  level  of  satisfaction  or  dissatisfaction  and 
their  underlying  causes  should  be  reported  to  the  OSD.  These  metrics  should  be 
benchmarked  against  trends. 

Turnover  statistics  should  be  reported  annually  and  expressed  in  terms  of  the  per¬ 
cent  of  primary  participants  (by  population)  assigned  to  positions  supporting  the 
DBS  at  the  beginning  and  end  of  the  previous  year.  The  data  should  be  derived  from 
unit  manning  documents.  This  metric  should  be  benchmarked  against  trends.  As 
earlier,  primary  participant  satisfaction  is  a  leading  indicator  of  turnover 
rates;  a  decrease  in  the  former  affords  the  opportunity  to  apply  interventions  de¬ 
signed  to  stem  the  latter  (particularly  if  the  cause  of  dissatisfaction  can  be  identified). 


Metrics  for  the  Cost  Per  Medical  Board  Decision  Output  Measure 

We  recommend  total  system  cost  divided  by  total  Informal  and  Formal  PEB  decisions 
rendered  as  the  metric  for  assessing  the  cost  per  medical  board  decision  output  mea¬ 
sure.  Military  departments  should  report  this  metric  to  the  OSD  annually,  based  on 
the  obligated  resources  and  the  medical  board  decisions  rendered  in  the  previous 
year. 

The  total  system  cost  should  be  derived  from  budget  data;  the  number  of  medical 
board  decisions  rendered  should  be  captured  from  an  automated  system  or  from  a 
hard  copy  form  accompanying  the  medical  boards.  We  recognize  that  this  metric 
does  not  provide  an  entirely  accurate  characterization  of  the  cost  per  medical  board 
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decision.'^  However,  in  the  absence  of  dramatic  changes  in  obligated  resources  or  in 
the  number  of  medical  board  decisions  rendered  in  a  particular  year,  the  metric 
provides  a  reasonable  indicator  of  cost  per  medical  board  decision.  This  is  the  pri¬ 
mary  reason  we  recommend  an  annual  report,  as  opposed  to  more-frequent  reports. 
Trends  should  be  the  benchmark  for  this  metric  with  the  objective  of  continually  de¬ 
creasing  the  cost  per  medical  board  decision  over  time. 

Metrics  for  the  Percent  of  Primary  Participants  Certified  Outcome  Measure 

We  recommend  the  following  metrics  for  assessing  the  percent  of  primary  partici¬ 
pants  certified  outcome  measure:  (1)  percent  of  commanders’  letters  submitted  by 
number  of  certified  commanders;  (2)  percent  of  medical  boards  dictated  by  number 
of  certified  physicians;  (3)  percent  of  PEBLOs  certified;  (4)  percent  of  Informal  PEB 
members  certified;  (5)  percent  of  Formal  PEB  members  certified;  and  (6)  percent  of 
PEB  administrative  action  officers  certified. 

As  noted  in  Chapter  5,  we  believe  that  certification  of  the  primary  participants  is  key 
to  accomplishing  the  purpose  of  the  DES.  As  a  result,  these  metrics  should  be 
benchmarked  against  both  trends  and  demanding  DoD  standards.  We  recommend 
the  military  departments  provide  these  metrics  quarterly. 

As  also  noted  in  Chapter  5,  we  recommend  that  commanders  and  physicians  become 
certified  through  their  respective  just-in-time  distance  training  packages  available 
from  an  OSD  Web  site.  This  will  provide  the  opportunity  for  nearly  100-percent  cer¬ 
tification  within  the  quarterly  reporting  time  frame.  Periodic  training  for  PEBLOs, 
PER  members,  and  administrative  action  officers,  if  scheduled  as  it  is  currently,  may 
result  in  lower  rates  of  certification  because  of  the  limited  scheduling  of  training 
opportunities.  Reporting  these  certification  metrics  quarterly,  however,  will  indicate 
whether  the  infrequency  of  training  opportunities  is  a  significant  problem  (as  it  may 
be  for  annual  training  and  high  personnel  turnover)  calling  for  an  intervention. 

For  the  first  two  metrics  for  this  outcome  measure,  the  commanders  submitting 
letters  and  the  physicians  dictating  boards  should  self-report  whether  they  are 
certified.  This  information  should  be  captured  in  an  automated  system  or  on  a  hard 
copy  form  accompanying  the  medical  board  when  it  arrives  at  the  Informal  PEB.  We 
found  that  incomplete  commanders’  letters  and  incomplete  narrative  summaries  are 
two  of  the  current  major  causes  of  delay  in  the  DES.  Associating  the  data  from 
specific  commanders  or  physicians  with  the  respective  medical  board  will  allow  the 
military  departments  to  assess  whether  noncertified  commanders  and  physicians 
materially  contribute  to  delays  in  the  system  and,  similarly,  whether  the  training 
leading  to  certification  is  accomplishing  its  purpose. 


“^PEB  decisions  rendered  on  medical  boards  in  a  particular  year  may  have  begun  in  a  previous  year  (and 
used  resources  obligated  for  that  time  frame).  Similarly,  the  PEBs  may  begin  considering  a  medical  board 
in  a  particular  year  but  may  not  render  a  decision  in  that  year  (and  will  use  resources  obligated  for  that 
time  frame  but  attributed  to  completed  cases).  Consequently,  significant  increases  or  decreases  in  this 
metric  in  a  particular  year  should  first  be  reviewed  in  terms  of  a  potential  anomaly  in  the  obligated 
resources  or  the  medical  board  decisions  rendered  in  that  year. 


210  Improving  Performance  of  the  DoD  Disability  Evaluation  System 


For  the  four  other  metrics  for  this  outcome  measure,  we  recommend  that  the 
military  departments  obtain  and  aggregate  the  data  from  personnel  records.  In  these 
cases,  individuals  act  collectively  to  process  medical  boards.  Consequently,  the 
overall  level  of  certification  is  a  more  important  measure  than  individual  certification 
data.  5 

Metrics  for  the  Total  Resources  Input  Measure 

We  recommend  the  following  metrics  for  assessing  the  sole  input  measure,  total  re¬ 
sources'.  (1)  number  of  individuals  broken  out  by  DES  phase  or  primary  participant 
population  devoted  to  the  DES;  (2)  pay  and  allowances  or  salaries  of  individuals 
broken  out  by  phase  or  primary  participant  population  devoted  to  the  DES;  (3) 
information  management  system  costs;  (4)  training  costs;  and  (5)  operations  and 
maintenance  costs  (other  than  training  and  civilian  salaries). 

Although  the  total  level  of  resources  is  important  as  an  indicator  of  the  resource  bur¬ 
den  the  DES  places  on  a  military  department,  how  those  resources  are  allocated  to 
the  various  phases  of  the  DES  influences  system  performance  more  directly.  We 
structure  the  metrics  in  this  area  accordingly. 

The  military  departments  should  report  total  resources  devoted  to  the  DES  by  major 
budget  areas  and  phases  of  the  system.  For  the  Medical  Evaluation  Board  phase, 
training  costs  are  the  key  metric.  For  the  PEB  phase,  the  metric  should  include  pay 
and  allowances  for  military  primary  participants,  civilian  salaries,  training,  and  in¬ 
formation  system  procurement. 

We  recommend  preparing  annual  performance  reports,  extracted  from  budget  data, 
for  both  the  previous  year  and  for  the  upcoming  budget  year.  These  metrics  should 
be  benchmarked  against  trends.  The  resources  allocated  in  the  budget  reflect  a 
commitment  to  future  performance  objectives.  Based  on  that  financial  commitment 
together  with  intended  interventions,  the  military  departments  should  provide  per¬ 
formance  objectives  for  other  performance  measures  (for  example,  productivity,  cost 
per  medical  board  decision,  percent  of  primary  participants  certified,  and  processing 
time). 

METRICS  RELATED  TO  THE  PROCESS  IMPROVEMENT  INTERVENTION 

Three  general  output  measures  capture  the  impact  of  actions  targeted  at  the  process 
improvement  intervention:  processing  time,  number  of  reworks,  and  time  to  pro¬ 
mulgate  policy  changes. 


^Because  PEBLOs  handle  individual  medical  board.s,  for  the  percent  of  PEBLOs  certified  metric  we 
considered  indicating  on  each  medical  board  whether  the  PEBLO  handling  it  was  certified.  We  did  not 
recommend  that  approach  primarily  because  a  PEBLO  may  become  certified  during  the  time  in  which  he 
or  she  is  handling  the  case,  confusing  whatever  impact  certification  may  have. 
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Metrics  for  the  Processing  Time  Output  Measure 

We  recommend  the  following  metrics  for  assessing  the  processing  time  output  mea¬ 
sure:  (1)  distribution  of  waiting  times  for  narrative  summary  dictation  (time  from  re¬ 
ferral  to  the  MTF  to  dictation  of  the  narrative  summary  or  to  the  service  member  be¬ 
ing  returned  to  duty);  (2)  distribution  of  waiting  times  for  Informal  PER  consideration 
(time  from  dictation  of  narrative  summary  to  Informal  PER  decision);  (3)  distribution 
of  waiting  times  for  Formal  PER  consideration  (time  from  Informal  PER  decision  to 
Formal  PER  decision);  and  (4)  distribution  of  waiting  times  for  decision  by  post-PER 
appellate  review  boards  (time  from  Formal  PER  decision  to  final  decision  by  highest- 
level  in-service  appellate  review). 

Although  average  processing  time  can  and  should  be  calculated  and  reported,  the 
distribution  of  waiting  times  provides  significantly  more  information  on  perfor¬ 
mance.  We  recommend  breaking  out  the  separate  phases  of  the  system  (including 
levels  within  phases,  such  as  the  Informal  and  Formal  PEE  levels  in  the  physical  dis¬ 
ability  evaluation  phase)  to  better  identify  the  potential  need  for  targeted  interven¬ 
tions.  The  waiting  times  should  be  computed  based  on  the  event  that  sends  a  medi¬ 
cal  board  to  the  next  phase  or  level  in  the  DES  process  without  regard  for  whether  the 
medical  board  is  returned  or  delayed  because  of  incomplete  information.^  (The 
metrics  associated  with  the  next  performance  measure — number  of  reworks — focus 
on  medical  boards  returned  to  an  earlier  phase  or  level  in  the  process.) 

The  processing-time  metrics  should  be  reported  quarterly  by  diagnostic  category 
based  on  medical  board  decisions  rendered  at  each  level  during  the  previous  quarter. 
They  should  be  benchmarked  against  trends  and  DoD  standards.  The  data  to  de¬ 
velop  these  metrics  should  be  captured  in  an  automated  system  or  on  a  hard  copy 
form  that  accompanies  the  medical  board. 

Metrics  for  Number  of  Reworks  Output  Measure 

We  recommend  the  following  metrics  for  assessing  the  number  of  reworks  output 
measure:  (1)  percent  of  deficient  commanders’  letters  at  Informal  PEEs,  by  reason; 
(2)  percent  of  deficient  commanders’  letters  at  Formal  PEEs,  by  reason;  (3)  percent  of 
deficient  narrative  summaries  at  Informal  PEEs,  by  reason;  (4)  percent  of  deficient 
narrative  summaries  at  Formal  PEEs,  by  reason;  (5)  percent  of  incomplete  medical 
boards  at  Informal  PEEs,  by  reason;  and  (6)  percent  of  incomplete  medical  boards  at 
Formal  PEEs,  by  reason. 

We  recommend  a  particular  focus  on  commanders’  letters,  narrative  summaries,  and 
medical  boards  because,  as  noted  earlier,  numerous  primary  participants  identified 
these  items  as  a  source  of  delay.  The  reasons  for  deficiencies  will  help  to  focus  on  the 
appropriate  interventions  to,  for  example,  modify  training  content  related  to  primary 
participant  certification.  We  do  not  recommend  employing  metrics  related  to  the 


®For  example,  the  time  period  from  when  a  medical  board  is  returned  to  the  MTF  from  the  Informal  PEB 
because  of  an  insufficient  narrative  summary  should  be  counted  against  the  waiting  time  for  Informal  PEB 
consideration. 
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amount  of  delay  that  reworks  cause  because  the  focus  should  be  on  eliminating  the 
need  for  reworks,  regardless  of  how  long  a  particular  rework  delays  the  overall 
process. 

The  reworks  metrics  should  be  reported  quarterly  based  on  medical  board  decisions 
rendered  at  each  DES  phase  and  level  during  the  previous  quarter.  They  should  be 
benchmarked  against  trends.  The  data  to  develop  these  metrics  should  be  captured 
in  an  automated  system  or  on  a  hard  copy  form  that  accompanies  the  medical  board. 


Metrics  for  the  Time  to  Promulgate  Policy  Changes  Output  Measure 

We  recommend  the  following  metrics  for  assessing  the  time  to  promulgate  policy 
changes  output  measure:  (1)  time  to  transmit  information  to  the  field;  (2)  time  to  up¬ 
date  military  department  policy  documents;  and  (3)  time  to  update  training.  These 
metrics  primarily  address  the  source  of  variation  among  military  departments. 

Rather  than  suggesting  the  military  departments  report  these  metrics  at  fixed  inter¬ 
vals,  we  recommend  that  the  military  departments’  Secretariat  send  a  letter  confirm¬ 
ing  the  promulgation  of  policy  in  each  of  these  three  metrics.  The  OSD  should 
develop  the  target  metric  for  promulgating  policy  based  on  military  department 
responses.  These  time  to  promulgate  policy  change  metrics  should  be  benchmarked 
against  military  department  trends  and  the  DoD  average. 


Appendix  I 

COST-BENEFIT  ANALYSIS  DATA 


This  appendix  contains  military  department  data,  shown  in  Tables  I.l  through  1.7, 
that  were  used  to  develop  the  training  cost-benefit  analysis  discussed  in  Chapter  7  of 
this  report. 
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7 he  Disability  Evaluation  System  (DES)  is  a  management  tool  used  by 
the  Department  of  Defense  to  determine  the  disposition  of  a  service 
member  who  has  a  medical  condition  that  calls  into  question  the  member's 
ability  to  perform  his  or  her  duties.  The  DES  exists  to  evaluate  service 
members  with  such  medical  conditions,  remove  those  unable  to  fulfill  their 
duties,  and  determine  a  disability  rating  for  those  who  are  removed.  This 
book  focuses  on  four  major  research  tasks  related  to  improving  system 
performance:  developing  a  basis  for  assessing  DES  outcomes,  identifying 
issues  of  variability  in  DES  policy  application,  conducting  a  DES  training 
analysis  and  presenting  recommended  changes,  and  developing  a  method 
for  continuously  monitoring  DES  performance.  As  part  of  their  findings,  the 
authors  constructed  a  set  of  desired  system  outcomes  and  from  that 
framework  suggest  comprehensive  training  and  information  management 
interventions  to  improve  overall  system  performance.  The  authors  also 
identify  groups  of  primary  DES  participants  and  outline  the  specific  bodies  of 
knowledge  and  skills  the  participants  require  to  execute  disability  policy 
consistently  throughout  the  military  departments. 
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